What's this about?
τ^τ-Bench gives a coding agent the materials of a realistic client engagement — business records, an inherited codebase, a client with unstated requirements, production APIs and serving budgets — and scores the customer-service agent it delivers against unseen simulated users. Claude Opus 5 under Claude Code led at 23.9%; the expert reference scored 82.2%.
What exactly is “end-to-end agent construction”?
Most benchmarks test whether an already-built agent can answer or act. This one tests the developer: understand the client's real need, ask questions, design the system, implement it, control cost and validate it. It is closer to delivering a commissioned software project than solving a coding puzzle.
Why is this worth noting?
The failure pattern is practical: shallow queries instead of understanding records, little communication with the client, too few architecture experiments, and shipping the first design that runs. Those are exactly the gaps hidden by benchmarks with clean requirements and a ready-made harness.