AI article
Three runs is not a sample size. It is a decision rule — and its error is exactly computable.
Community description: Three runs, or five seeds, is a convention, not a computation. The repeat axis of stochastic evaluation has an exactly computable error law — and the item-vs-repeat budget boundary that follows from it.
Dev.to | Oct 13, 2026 | howcani howcani
Automated excerpt
Grab any paper that compares two models, two agents, two prompts. Unanimity gets worse as you add runs. 0. 2710 at N = 3, 0. 6513 at N = 10. And at equal cost, an even N beats the odd N above it: at p = 0. 10, 0. 0280 for N = 3, 0. 00370 for N = 4, 0. 00856 for N = 5.
Selected automatically from source text; not independently written or fact-checked. Read the original for full context.