What's this about?
A video-generation model (5B and 14B sizes) using hybrid linear attention with attention residuals to match full-attention video transformers in quality while being efficient enough to generate 720p video on a single GPU (28 upvotes).
What exactly is "linear attention"?
Standard attention compares every frame-patch with every other, and the cost explodes with video length. Linear attention approximates this at a cost that grows proportionally instead — the difference between everyone-shakes-hands-with-everyone and passing one message around the room. The "residual" part patches back the quality the approximation loses.
Why does it matter?
Video generation has been gated by data-center hardware; single-GPU 720p moves it toward the local, hobbyist territory where image generation exploded in 2023.