What's this about?
"We ran Astra on ExploitBench where the model achieved a perfect score of 100% on the benchmark to evaluate the model's ability to develop exploits from known vulnerabilities."
What is the gap between a vulnerability and an exploit?
A vulnerability is a flaw — the program does something wrong given certain input. An exploit is a working weapon built from it: reliable control of execution, past whatever mitigations the platform has, producing a specific outcome. For memory-corruption bugs in something like a browser engine, that gap is where most of the skill lives; a bug report saying "this crashes" is worlds away from "this executes my code." A 100% score is a claim about closing that gap consistently.
Why does this matter to you?
- A saturated benchmark is a benchmark that has stopped measuring — which is exactly why OpenAI immediately built a private one, described below.
- Read against Anthropic's line the same day: Fable 5.1 may discover vulnerabilities but not develop exploits. Two labs, one capability, opposite policies.
- The gap between "found a bug" and "has a working exploit" was, until recently, the main thing limiting how fast disclosure turned into attack.