WebLLM

What's this about?

A high-performance in-browser LLM inference engine — running a language model entirely in a web page, with no server call.

How does a browser run a model at all?

Through WebGPU, which gives JavaScript access to the machine's graphics hardware. The model weights are downloaded once and cached, then inference runs locally on your GPU. The consequences are the interesting part: no API key, no per-token cost, no network round trip, and — the one that matters most — the text never leaves the machine. The limits are equally clear: you are constrained to models small enough to download and fit in browser memory.

Why is it worth noticing?

1. It is the strongest available privacy story for any application handling sensitive text.
2. It is the same trend as the local-model setups and the 104GB-on-48GB experiment from yesterday, at the other end of the size range.
3. Low comment count for the score usually means people bookmarked it rather than argued about it.