What's this about?
"Poor Lab's Qwen2-1.5B trained on RTX 5090 within $5090" — a cost-efficient, hardware-accessible pretraining recipe on a single consumer GPU. 28 upvotes.
Why does pretraining cost so much normally?
Because you are processing trillions of tokens through a model that does not yet know anything, on hardware that costs more per hour than most people's monthly compute budget. Fine-tuning an existing model is cheap; pretraining from scratch is the part that has been out of reach for academia and hobbyists, which means nobody outside a lab can study what happens *during* it. That is the gap this targets — not producing a competitive model, but making the process reproducible by people who want to understand it.
Why is it worth noticing?
- The title's arithmetic joke is the pitch: one RTX 5090, under $5,090.
- It is the same instinct as the small-models argument from last week — capability per dollar, not capability per parameter.
- Reproducibility of pretraining is a genuine research bottleneck; open weights don't help if you can't rerun the recipe.