What's this about?
In this episode Dwarkesh Patel lays out eight predictions for a shift away from today's train-once-then-freeze models toward systems that keep learning after they're deployed — "continual learning."
What is "continual learning"?
Today's models are trained on a fixed dataset, then frozen — they don't learn from their day-to-day use (that's why they forget your last conversation). Continual learning is the goal of models that keep updating from new experience without catastrophically forgetting old skills. It's long been a hard, unsolved problem in ML.
Why does it matter?
If models can learn on the job, they get personal and improve with use — but it also reshapes safety, evaluation, and business models (a model that changes after release is harder to certify). The predictions are a useful map of where a lot of researchers think the field is heading.