What's this about?
OpenAI published test results for Jalapeño, its first in-house inference chip, claiming more intelligence per watt and both higher throughput and lower latency at once — usually a trade-off. SemiAnalysis followed with an independent teardown headlined that it beats Nvidia's Blackwell.
What exactly is an "inference chip"?
Training is teaching a model; inference is running it to answer your questions. They stress hardware differently — training wants raw parallel maths, inference wants to move a lot of memory quickly at low latency. A chip built only for inference can drop the training-oriented circuitry and win on efficiency, which matters because inference is the part you pay for every single day.
What's the community saying?
HN 346 points, 241 comments
- The most-replied idea: if you have this much scale, why not bake specific model weights directly into silicon — a $100M chip run that is 10x faster and cheaper would pay for itself.
- Genuine surprise at a figure in the analysis: humans are still about 22x more energy-efficient than this hardware per token of speech, which several read as closer than expected.
- Scepticism about manufacturing: how does OpenAI mass-produce more economically than Nvidia, which owns the supply-chain relationships and scale?
- The optimistic read on prices — continued hardware gains make it hard to believe token costs won't keep falling.
- A meta-thread appreciating that world-class trillion-dollar-industry analysis now comes from SemiAnalysis rather than traditional analyst firms.