What's this about?
The loudest weekend conversation wasn't about Opus 5 itself but what its scores mean: a viral thread (2.9k likes) argued that anyone who actually uses both models can tell Opus 5 is "nowhere near" Fable 5 in practice — yet it beats Fable on paper, which the author takes as proof that general benchmarks have stopped measuring anything real. OrangeAI pushed it further: if optimizing for verifiable rewards beats optimizing for human preference, "did the alignment plan fail?"
What exactly is a "benchmark"?
A benchmark is a standardized exam for AI models — a fixed set of problems (code to fix, questions to answer) with a score at the end, so models can be compared apples-to-apples. The trouble is the same as with human exams: once everyone teaches to the test, high scores stop guaranteeing real-world competence. When a cheaper model outscores a stronger one that users clearly prefer, either the cheap model is underrated or the exam is broken.
What's the community saying?
- The practical-use gap: "a few tasks is all it takes to feel the difference" — benchmarks reward test-shaped work, not ambiguous real work.
- Mollick's nuance from pre-release access: Opus 5 matches Fable on short tasks but is "less ambitious" on long ones — a dimension no current benchmark captures.
- The RLVR-vs-alignment worry: if human preference no longer decides which model "wins", the industry's scoreboard and its stated values have diverged.
- A counter-camp: maybe users are the miscalibrated ones, anchoring on style quirks ("FableSpeak") rather than output quality.