What's this about?
The community assumed hybrid models' recurrent layers had to stay at 8 or 16 bits — recurrence should accumulate quantization error. This paper (46 upvotes) quantized all 496 linear layers of a Gated DeltaNet model to 4 bits and matched BF16 within seed noise.
What exactly is quantization?
Storing weights with fewer bits — 4-bit halves memory again versus 8-bit, letting bigger models fit on smaller hardware. The intuition being corrected: errors in a loop compound. Empirically, here, they don't.
Why is this worth noting?
An empirical result overturning a folk belief nobody had tested — the community left performance on the table out of caution. Practical upshot: hybrid/recurrent architectures are cheaper to deploy than assumed.