What's this about?
ARC Prize published its own account of Astra's ARC-AGI-3 result, and the number everyone quotes — 99.9% — is from OpenAI's "Provider Adapter" harness, which lets the model use its own note-taking and context management. On ARC's neutral Standard harness, the identical model scores 62.7%. ARC has decided to publish both numbers going forward, and states flatly that saturating the benchmark is not proof of AGI.
What exactly is a "harness"?
The harness is everything wrapped around the model: how the task is presented, what tools and memory it gets, how many retries it's allowed, how its output is parsed. Same engine, different car. A 37-point swing from harness alone is the largest public demonstration yet that "which model is best" is often really "which harness was used" — and it lands the same week as papers arguing the harness now matters as much as the weights.
What's the community saying?
- The philosophical push-back: "is solving a snake-like puzzle game in the least number of moves really what defines intelligence?"
- The goalpost prediction: fine, 99.9% with the right harness means AGI by this test — so the goalposts will now move to cost-efficiency, and the cycle will repeat forever.
- Amazement at the economics: the score/cost curve shows runs costing tens of thousands of dollars, with commenters asking who pays for this — while ARC notes human participants cost about $12.78 per attempted game.
- A methodological worry: did OpenAI have prior access to ARC-AGI-3 tasks, enough to tune a harness for them?
- One overlooked number: with *no reasoning at all*, Astra scores 35.2% on the standard harness — above where whole model families were last year.