An LLM on an $8 microcontroller

What's this about?

A hobbyist got a 28.9-million-parameter language model generating text on an ESP32, the $8 Wi-Fi chip found in smart lightbulbs and DIY electronics. It's thousands of times smaller than frontier models, but it runs entirely on hardware that costs less than lunch.

What exactly is an "ESP32"?

The ESP32 is a microcontroller — a fingernail-sized computer with a few hundred kilobytes of working memory, no operating system, no graphics card. It's what powers cheap IoT gadgets. Frontier models need warehouse GPUs; squeezing any coherent language model into this budget is an exercise in extreme compression.

What's the community saying?

1. Definitional fight: at 28.9M parameters, is this an "LLM" or just a language model? (The first L stands for Large.)
2. Genuine use-case hunting: offline voice commands, toys, and privacy-preserving appliances that never phone home.
3. Appreciation for the engineering writeup itself — quantization and memory-mapping tricks — independent of practical value.