What's this about?
Discovery — formulating novel generalizations — is central to science, yet few AI benchmarks probe it directly. DiG-bench measures that capacity inside games.
What exactly is "discovery" as distinct from problem-solving?
Solving a problem means finding the answer to a question someone posed. Discovery means noticing that a question exists and that some previously unremarked regularity holds. A model given a game can learn to play it well; discovering that a hidden rule governs the board is a different act. Games are a good testbed because the underlying rules are fully known to the evaluator but concealed from the model, so you can check whether a claimed generalization is actually true.
Why is this worth noticing?
- Most benchmarks reward finding a known answer, which is precisely the skill that saturates and stops being informative — measuring discovery targets what remains genuinely hard.
- It connects to yesterday's Gowers post on mathematics: his test for real capability was models producing methods that are new and surprising, not findable by brute search. DiG-bench is an attempt to measure something in that family.