What's this about?
Coding agents have got good at bug fixing. Can they do a long-horizon, whole-repository stack migration — the kind of technical-debt work that accumulates over decades and is currently almost entirely manual? Existing benchmarks can't answer this because they only evaluate behavioural correctness. 13 upvotes on HuggingFace.
What is a "stack migration"?
Moving an entire codebase from one foundation to another: a framework version bump that changes every call site, a language migration, replacing a database layer. What makes it hard is not any single edit but that thousands of edits must be mutually consistent and the system must work at the end, not at each step. It is the canonical task where a human takes weeks and an agent's context window becomes the binding constraint.
Why does this matter to you?
- "Behavioural correctness isn't enough" is the right critique — a migration can pass every test and still leave the codebase worse.
- Long-horizon whole-repository work is where compaction (see the Compaction Cliff) bites hardest.
- This is the benchmark shape that matters if you want agents doing maintenance rather than features.