What's this about?
A study of what happens when a user's goal shifts mid-conversation — as real goals always do — finding models anchor on the original framing and degrade as intent evolves (21 upvotes).
What exactly is "evolving intent"?
People rarely specify what they want upfront; they discover it through interaction ("actually, make it a dashboard instead"). The benchmark measures whether models track the current goal or keep serving a stale one.
Why does it matter?
It quantifies the most common real-world agent frustration — having to start a fresh chat because the model won't let go of an earlier version of the task — and gives labs a target to train against.