What's this about?
Epoch launches Benchmark Reviews with 15 audits: four verified, nine flawed and two lacking enough information. The labels describe test quality—not a new ranking of model intelligence.
What exactly is benchmark audit?
Checking whether a test's questions, scoring and procedures measure what they claim. A flawed benchmark can mis-rank models even when its headline score looks precise.
Why does this matter?
Original X: 736 likes, 42 replies; Mollick praises the audit initiative.
- Significance: saturated and erroneous tests obscure progress.
- Significance: quality labels must be read with review methods.
- Significance: an audit is not proof a model is reliable in your workflow.