Holdout Ledgers Keep Agent Scores Honest

Automated excerpt

Difficulty labels belong on the ledger before agents run, because labeling after seeing failures turns the scale into a story about the winner. Readers can reject the ranking when the ledger hash, the harness commit, or the baseline identifier is absent. A methodology that publishes the ledger digest, the harness commit, the baseline identifier, and the spent budget cannot hide extra retries.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.