What's this about?
Big Pineapple, the platform behind 1.1.1.1, holds over 250 billion DNS cache entries at any moment — at that scale a single wasted byte per entry costs 250GB across the fleet. Five successive changes to how entries are stored cut the per-entry footprint by more than half, freeing roughly 100 terabytes of RAM, equivalent to 130 of their servers. The cache also got faster: insert throughput up 43%, lookup latency down 19%. One of the wins was noticing that cached responses are never modified after being written, so the 8-byte capacity field every growable vector carries was pure waste.
Why does "never modified after writing" save memory?
A growable array has to remember three things: where the data is, how much is in it, and how much room it has. That third number only matters if you might add more later. A DNS response is written once and read forever, so the capacity field is 8 bytes of bookkeeping for an event that never happens. Trivial per entry; 2 terabytes at 250 billion entries.
What's the community saying?
- The first question is why nobody caught something this trivial in design review, answered simply: "it was working so no one thought to check."
- That launches a real argument about premature optimisation, with Cloudflare's own CTO responding that if you aren't memory constrained, minimising memory is a genuine waste of time when you need to ship.
- The economic update several people make: memory is now up to 10x more expensive, which changes the calculation for anything with a large footprint.
- The most forward-looking comment: LLMs have shifted the equation, because "write a harness, profile it, build experiments, profile those, recommend one" is exactly the bounded goal-directed job models are now good at.
- A practical defence of the delay — good systems engineers trained on your codebase are expensive, and this simply never reached the top of the list.