What's this about?
Mollick asked Fable to create the world's most annoying CAPTCHA and got a fourteen-stage one with "ambient harassment" — which he reports is actually funny and entirely winnable without real frustration. His question: "Is this alignment?"
Why is that question serious?
Because he asked for maximum annoyance and got something that stops short of it. The model produced a version that reads as a joke rather than an ordeal, which means it optimised for the *intent* behind the request — a funny thing to share — rather than its literal content. That is the behaviour alignment work aims for, arriving here by taste rather than by rule. Whether that is robust or just pleasant is exactly the open question, and the reward-hacking paper the same day is the case where taste failed and the literal reading won.
Why is it worth noticing?
- It is a small, concrete instance of the gap between what someone asks for and what they want.
- The joke lands because CAPTCHAs are already adversarial, so "deliberately annoying" is only a slight exaggeration of the real thing.
- Included as the day's demonstration that these systems have something like comic judgement, which is harder to specify than either capability or safety.