The Judge Reads Tokens: An Adversarial Evaluation of an Eval Toolkit's Own Scorers

Automated excerpt

This repo's philosophy says: deterministic first, judge second, and when you use a judge, measure the judge. Anyone tuning output length against this judge is optimising against two staircases. Judge validation collapses off the benign distribution.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.