What's this about?
"Soup" fine-tunes an 8-billion-parameter model on a 4GB consumer GPU — training, not just running (HN 126 pts, 24 comments).
What exactly is fine-tuning, vs. inference?
Inference is using a trained model; fine-tuning is further-training it on your own data to specialize it. Fine-tuning is far more memory-hungry (it stores gradients and optimizer state), so doing it in 4GB is the notable feat — yesterday AirLLM did *inference* on 4GB, today Soup does *training*.
What's the community saying?
HN 126 pts, 24 comments.
- The democratization angle: private, custom models trained on hardware people already own.
- The usual "but how slow / how good?" — memory tricks (offloading, low-rank, quantized optimizers) trade speed and quality for fit.
- Read together with AirLLM, a clear week-over-week theme: shrinking both training and inference onto tiny GPUs.