What's this about?
Generation before reconstruction, for stable end-to-end latent generative modelling. 59 upvotes.
What is the ordering being reversed?
The standard latent generative pipeline learns to compress data into a latent space first — the reconstruction objective — then learns to generate within it. The compression is trained to preserve what a reconstruction needs, which is not necessarily what generation needs, and the two objectives can pull the training apart. Putting generation first means the latent space is shaped by what the generator requires. "Stable" in the title is the honest signal: this is about training runs that converge, which is a large fraction of the real difficulty.
Why is it worth noticing?
- Training stability is under-published relative to how much of practitioners' time it consumes.
- Ordering-of-objectives results tend to generalise across architectures.
- 59 upvotes for something this foundational suggests the instability is widely felt.