What's this about?
The authors formalise prefix invariance — the representation at position t must not depend on inputs that come after it — and give a lightweight audit needing two forward passes, no training and no gradients, that localises exactly where a model violates it. 29 upvotes.
What exactly is "prefix invariance"?
When a model generates text left to right, it must not be able to see the future. Architectures enforce this with a mask. The paper's point is that the mask is a stated intention, not a proof: in attention, state-space and hybrid models, implementation details can leak future information anyway — which would quietly inflate benchmark results.
What's the community saying?
The appeal is the cost: two forward passes with no training means anyone can run this against their own model today. Cheap diagnostics that catch silent correctness bugs tend to spread faster than better architectures.