What's this about?
The problem statement is unusually blunt: because attention is quadratic, LLMs consume enormous memory and energy, every new token costs more than the last, and for each additional token the keys and values must be stored in memory indefinitely — which the paper calls unsustainable. Several alternatives exist; this one argues a plain sliding window outperforms linear attention. 12 upvotes.
What is the trade being made?
Full attention lets every token see every previous token — maximum capability, quadratic cost. Linear attention approximates that with a compressed summary at linear cost, and the approximation loses things in ways that are hard to characterise. A sliding window just... doesn't look back beyond N tokens. It is the least clever option, exactly implementable, and the claim is that for the same budget it beats the sophisticated approximation. That is a familiar shape: the elaborate method costs capability elsewhere to buy back what the simple constraint gives up honestly.
Why does this matter to you?
- Another entry in this month's run of papers checking whether the fashionable method beats the boring baseline.
- Memory cost per token is now an economic question, not just an engineering one — see the memory shortage story in the same digest.
- If it replicates, it simplifies a lot of architectural decisions in the wrong direction for people who have already built the complicated thing.