Gemma 4 26B on a 13-year-old Xeon

What's this about?

Someone ran Google's Gemma 4 26B model at 5 tokens/second on a 13-year-old Xeon server with no GPU at all — proving that decent local AI is possible on hardware most companies threw away years ago.

What exactly does "5 tokens/sec, no GPU" mean?

A token is roughly ¾ of a word; 5/second is about human typing speed — slow but usable for background tasks. LLMs normally demand expensive GPUs; running one on an ancient CPU works because of quantization (shrinking the model's numbers to fewer bits) and the memory-bandwidth-bound nature of inference. Total hardware cost: essentially scrap value.

What's the community saying?

1. E-waste-tier hardware as a real inference option — several share their own old-server setups.
2. Debate over whether 5 tok/s is "usable" — fine for agents running overnight, painful for chat.
3. The broader open-weights point: this is only possible because Gemma's weights are downloadable — connecting it to the day's Inkling story.