What's this about?
Replicability underpins scientific reliability, and attempting a replication routinely surfaces details the original paper left unstated. This work trains AI agents to perform replication.
What exactly is a "replication"?
Replication means taking a published paper and independently reproducing its results from the described method. It sounds mechanical and is notoriously not: papers omit hyperparameters, preprocessing steps, and the small decisions that made things work. That's precisely why replication is valuable — the attempt exposes what the paper actually depended on. It's also thankless work that rarely advances a career, which is why so little of it happens.
Why is this worth noticing?
- It targets a genuine bottleneck in science, where the shortage is human attention rather than ideas, and where the work is well-defined enough for an agent to attempt.
- Replication has a built-in success criterion — you either reproduced the number or you didn't — which makes it a rare research task with an honest automatic grader.