What's this about?
Dwarkesh Patel lays out predictions for a coming shift away from today's "train once, then freeze" models toward models that keep learning after they're deployed. Worth pairing with today's Macaron-V1 paper, which is a concrete stab at exactly that.
What exactly is "continual learning"?
Current big models are pretrained once and then frozen — they don't remember your last conversation or get smarter from using a tool. Continual learning means a model updates itself from ongoing experience after deployment, without catastrophically forgetting what it already knew. It's hard because naive updating tends to overwrite old skills.
Why is this notable?
If it works, it changes the economics and shape of AI products — models that improve in the field rather than only at the next big training run. It's one of the more credible "what's next after scaling" theses.