What's this about?
Generating audio and video together over long durations, for persistent stories and interactive worlds — that is, content that holds together over minutes rather than seconds, with characters and settings that stay consistent. 19 upvotes.
What makes "long-horizon" hard?
Drift. Every generated second is conditioned on the previous ones, so small inconsistencies compound: the character's jacket changes colour, the room gets a window it didn't have, the voice shifts register. Over ten seconds nobody notices; over three minutes it is unwatchable. Fixing it means giving the model persistent state about what has already been established, which is a memory problem wearing a rendering problem's clothes.
Why does this matter to you?
- Gemini Omni 1.1 Flash's headline feature today is scene extension to 40 seconds using 10 seconds of context — the same problem, attacked commercially.
- "Interactive worlds" is the point where video generation meets world models, and where the compute costs become interesting.
- Audio-visual together is harder than either alone, because desynchronisation is immediately perceptible.