What's this about?
A Hot Chips 2026 writeup of Samsung's Processing-in-Memory work — putting compute units inside the memory itself rather than moving data to a processor.
Why would you compute inside memory?
Because for a lot of modern workloads, moving the data costs more than the arithmetic. An LLM generating tokens spends most of its time reading weights out of memory to do a small amount of maths on each one; the multiplier is nearly free and the transfer is not. Putting simple compute next to the memory banks means the data barely moves, which attacks the actual bottleneck. The catch is that "simple compute" is doing real work in that sentence — it only helps for operations that fit the constrained units available.
Why is it worth noticing?
HN 278 points, 106 comments
- Memory bandwidth is the binding constraint on inference, and memory prices are up sharply — both make this more interesting than it was two years ago.
- Hot Chips coverage from Chips and Cheese is consistently the most technically substantive available.
- Adjacent to the Cloudflare memory-optimisation story from earlier in the week: the industry is paying attention to memory again because it got expensive.