What's this about?
HumanLayer published a benchmark that scores coding models not on whether they pass a test today, but on the longitudinal, non-functional quality of the code they produce — the "slop" that accumulates and makes a codebase hard to change later. Applied to Opus 5, it became the calm counterweight to launch hype: experienced users report Opus 5 is a genuine step up from Opus 4.8 but not the leap Fable felt like, and several now run Opus-5-medium in place of Opus-4.8-xhigh to save tokens.
What exactly is "slop"?
"Slop" is plausible-looking output that's subtly low-quality — code that works in the demo but is bloated, over-abstracted, or rewrites things it shouldn't, so the next change is harder. Like a contractor who finishes the room fast but leaves wiring you'll pay for later. Most benchmarks only check "does it run"; this one tries to price the future maintenance debt.
What's the community saying?
- "This matches my experience" — Opus 5 as a nice-not-revolutionary upgrade; token savings by dropping to a lower effort tier.
- Sharper dissent: "Opus 5 is an overconfident model that generates too much slop" — some reverted to Fable and Codex Sol.
- The harness question: is the slop the model's fault or the system prompt's? Maybe a "check your work after finishing" instruction fixes much of it.
- Coverage gaps flagged: no GPT-5.6 / GLM 5.1 / Kimi K3 runs yet (cost), so the comparison is partial.