What's this about?
Today's top HuggingFace paper (216 upvotes) attacks AI agents from an angle most safety benchmarks miss. Instead of writing malicious prompts, it evolves the shared environment the agent works in — the files, records and state it repeatedly reads and modifies. OpenART supplies over 10,000 validated stateful scenarios across 50 domains, drawn from a pool of more than 500,000 tools and skills, with a median of 97 tool calls per task, evaluated across 75 agent-model configurations. Its attack policy, EMHA, reaches a pooled 85% attack success rate, and its advantage over prompt-only attacks grows from about 2% in simple environments to over 17% in the most complex ones.
What exactly is "red teaming"?
Red teaming means employing people — or here, an automated system — to attack your own product before someone hostile does. The name comes from military exercises where a red team plays the adversary. For AI agents it means deliberately trying to make the agent do something harmful, then fixing whatever worked. The contribution here is that the attacker never sends a malicious instruction; it only makes authorized changes to the environment state, which is much harder to filter because nothing in the input looks like an attack.
Why is this worth noticing?
- Task objectives stay fixed while only the environment changes, which isolates a real and under-tested failure mode: an agent that behaves correctly on a clean system can be walked into harm by state left behind earlier.
- The finding about runtime implementation is the most consequential part. The paper reports that how an agent is implemented — its harness — explains a significant share of safety variation beyond the underlying model's capabilities. That means safety is a property of the harness, not just the model, which lands the same day DeepSeek open-sourced a harness and The Economist called harnesses "barbed wire."
- The scale changes what's measurable. A median of 97 tool calls per task is far beyond the short, static interactions most safety evaluations use, and the attack advantage growing with complexity suggests today's benchmarks systematically understate risk.