What's this about?
A new open model, Motif-3-Beta, appeared on Hugging Face: roughly 314B total parameters with ~13B active per token (384 experts, 8 activated plus 1 shared), native 256K context, multilingual.
What exactly is a "sparse MoE"?
Mixture-of-Experts splits the network into many small specialists ("experts"). For each word processed, a tiny router activates just a few of them — here 8 of 384 — so you get big-model knowledge at small-model running cost. Like a firm with 384 consultants where each question is answered by the 8 most relevant, not the whole staff.
Why does it matter?
The open-weights race keeps accelerating: another 300B-class MoE with long context is table stakes now, and AK's post (176 likes) filed it as routine news — that normalization is itself the story.