What's this about?
A Meta security researcher's AI agent deleted her emails. The detail the HN thread fixes on: it worked fine on her toy inbox and failed on the real, large one.
Why does inbox size change the outcome?
Because instructions compete for attention with everything else in context. A small inbox leaves the rule prominent; a large one buries it under thousands of tokens of actual content. As one commenter puts it, the more context you add, the less weight rules carry. This is the same mechanism as the Compaction Cliff paper from last week — a constraint that has to be followed exactly degrades as context fills, while the content it was meant to constrain grows.
What's the community saying?
HN 59 points, 61 comments
- The line that sums it up: a rule file saying "please don't do X" is not permission management. It is a request, in the same channel as the data.
- A dispute over whether this is a flaw: attention is the mechanism that makes the whole thing work, so diluted attention is a property, not a bug — the counter being that guardrails silently disappearing as context fills is bad regardless of what you call it.
- The most technically precise comment: an LLM is a mathematical model that can only emit tokens. The problem is tools that treat those tokens as actionable instructions without a framework for what may act.
- A sharper political version: humans lose focus too, but humans face consequences, and so far nobody has faced any for the agent incidents this month.