What's this about?
A position paper arguing that agent release decisions still rely on capability benchmarks while production readiness — reliability of retrieval, state management, tool calls, and failure recovery — goes largely unmeasured, and proposing what a readiness evaluation should cover.
What exactly is "capability vs. readiness"?
Capability asks "can it do the task once, in a demo?" Readiness asks "does it keep doing the task correctly at 2 a.m., on weird inputs, when an API times out?" It's the difference between a student passing an exam and an employee you'd leave alone with the production database.
Why it matters
As agents move into real workflows, the gap between benchmark scores and operational reliability is where the incidents will come from — this gives teams vocabulary to push back on "the benchmark says ship it."