What's this about?
Enclave reports 11/11 vulnerable targets compromised and four fixed controls intact, at $4.65 for accepted runs ($5.14 including retries); five successes used unintended benchmark routes, since repaired.
What exactly is benchmark shortcut?
An agent achieves the measured outcome by a route the test designer did not intend. This may show useful ingenuity, but comparisons must use identical repaired benchmark versions.
What's the community saying?
HN 159 points, 66 comments.
- A “best hacking model” claim needs wider matched comparisons.
- Some practitioners report better results from GLM on their own targets.
- Cached input explains the low bill; real cost and task difficulty still matter.