← Back to digest
Commentary

"It's the large models rolling the dice"

What's this about?

"I don't know whether God plays dice, but right now it's the large models rolling them — and humans have already started making money off those rolls." dotey's reply: does swapping "rolling dice" for "casting a hexagram" suddenly make it sound more scientific and reasonable?

What is the observation underneath the joke?

That these systems are sampling, not computing. A model with temperature above zero produces a different answer each time, drawn from a distribution — and an enormous amount of economic activity is now built on outputs generated that way. Einstein's objection to quantum mechanics was that a fundamental theory should not be probabilistic. The modern version is stranger: the randomness is not a limitation anyone is trying to remove, it is load-bearing, and the same prompt twice is not expected to give the same answer.

Why is it worth noticing?

160 likes on the original
  1. It is the funniest framing of the week for something the industry has largely stopped noticing.
  2. dotey's hexagram reply makes the sharper point: the vocabulary we use to describe sampling determines whether it sounds rigorous or mystical, and neither word changes the mechanism.
  3. Non-determinism as an accepted property of production systems is a genuinely new engineering situation.
← oran_ge publishes two podcasts