What's this about?
Baseten's Philip Kiely and Ali Taha on the engineering discipline of serving models fast and cheap — recorded as Baseten closes a $13B-valuation round on the back of the "inference inflection".
What exactly is "inference engineering"?
The specialty of making a trained model serve requests efficiently: batching, KV-cache management, quantization, speculative decoding. The gap between naive and expert serving can be 10× in cost — which at today's token volumes is the whole margin.
Why does it matter?
Pairs neatly with the Dwarkesh essay: he argues compute gets scarce, this episode is about the people squeezing more out of it. Also includes Ali's viral 48-hour teardown of Kimi K3's modeling code ("650mg of caffeine, 8 papers").