The Two MMLU Scores: What a Benchmark Name Does Not Fix

Automated excerpt

Two MMLU accuracies, 0. 781 and 0. 79, for two builds of one model family under the same benchmark name; for a score-delta query the verifier returns incomparable (Sections 1 and 5). mmlu fixes a name. Two evaluation records appear in the same table. That two identical identifiers name the same thing.

Selected automatically from source text; not independently written or fact-checked. Read the original for full context.