What's this about?
AI for games has focused on open-ended, fluid interactive storytelling, and this paper (26 upvotes) argues the field has largely ignored a critical challenge: maintaining long-horizon logical consistency. It introduces a benchmark for whether an agent stays coherent with what has already been established over a long interaction.
What exactly is "long-horizon consistency"?
It's the difference between saying something sensible and remembering what you said. Over a long interactive narrative, a model must keep facts stable — a character who died stays dead, a locked door stays locked unless someone unlocked it, an established motivation doesn't silently reverse. Models are strong at local fluency and weak at this kind of bookkeeping, because nothing in next-token prediction enforces continuity with something said two hours ago.
Why is this worth noticing?
- The failure mode generalizes well beyond games: any long-running agent — a research assistant, a customer-support agent, a coding agent on a multi-day task — needs exactly this property.
- It's measuring the unglamorous half of interactive AI. Open-ended generation demos well and consistency doesn't, which is plausibly why the gap existed.