What's this about?
Can a language model recover the true research idea of a published paper when given only that paper's pre-publication bibliography? Reconstruction is a blind idea-recovery benchmark that withholds the paper itself and provides only what its author had read.
What exactly makes this a clean test?
The contamination problem plagues research-capability benchmarks: models trained on the web have usually seen the papers you're testing them on, so apparent insight is recall. This design is clever because the bibliography is a genuine snapshot of the author's inputs before the idea existed. If a model can reconstruct the contribution from the same reading list, that's evidence of synthesis rather than retrieval — assuming, and this is the load-bearing assumption, the model hasn't seen the resulting paper.
Why is this worth noticing?
- It targets the most interesting version of "can AI do research" — not executing an experiment, but arriving at the idea worth pursuing given what was known.
- It has an unusually honest structure: there's a definite right answer (the paper that was actually written) without the answer being available in the inputs.
- It joins today's Apodex and this month's replication work as part of a shift from benchmarking answers toward benchmarking the research process itself.