Do LLMs Catch Bad Startup Math? My First Answer Was Wrong.

Automated excerpt

Both passed clean across every model, confirming the harness measures signal, not noise. I benchmarked six models across three labs using a paired frontier-vs-small-tier design — one flagship and one cost-optimized model per lab — to isolate whether model scale predicts arithmetic verification reliability, independent of which lab produced it. Frontier Consistency: All three frontier models scored 3/3 across both scenarios on every trial, with zero variance.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.