What's this about?
Yesterday this was a countdown teaser. Today the weights are out. It is a multimodal mixture-of-experts model and, explicitly, an early preview of the architecture Qwen4 will be built on — the same role Qwen3-Next played for Qwen3.5. Four changes: a Gated DeltaNet plus Qwen Sparse Attention hybrid, a Gated Residual that widens the residual stream into four branches, plus embedding and optimisation work. The published sizes are a 125B main model with an additional 51B of N-gram embeddings and 6B parameters active per token.
What exactly is "active parameters"?
A mixture-of-experts model contains many specialist sub-networks but only wakes a few for any given token. So a 125B model with 6B active costs roughly what a 6B model costs to run, while holding far more knowledge. It is why these models can be simultaneously large and cheap — and why the memory requirement stays high even though the compute drops, since all the experts must be resident.
What's the community saying?
- Genuine surprise at the quality jump: one commenter notes it beats the previous 27B cleanly, and that self-hosting something at this level on a laptop is a remarkable place to be.
- A practical warning that nothing runs it easily yet — llama.cpp and vLLM lack mainline support.
- Confusion about effective size, since 125B plus 51B of embeddings is ~176B, and it is unclear how the embedding portion quantises.
- Simon Willison ran his standard pelican test across four reasoning levels and reports not getting one he liked as much as the older model's output.
- A joke thread parodying the model's overthinking, written in the voice of a model second-guessing itself.
- Mild fatigue — "do we really need breaking news about Qwen every single day?"