What's this about?
AgentCompass proposes shared evaluation infrastructure for AI agents, attacking the fragmentation problem: today every agent benchmark ships its own tightly-coupled pipeline, making results hard to reproduce or compare. (HF 9↑)
What exactly is broken in agent evals?
Testing an agent means standing up environments, tools, and scoring — and each benchmark hardwires its own versions. Two papers reporting "70% on task X" may have measured different things. Unified infrastructure is the boring-but-critical plumbing: same harness, comparable numbers.
Why does it matter?
As agent claims drive purchasing and safety decisions, non-reproducible evals become everyone's problem. Infrastructure papers age better than benchmark-of-the-week papers.