What's this about?
ScienceIDE uses expert-defined acceptance criteria to convert repositories into executable agent-training environments, then trains 72B/9B/4B models on verified trajectories; gains are author-reported (70 HF upvotes).
What exactly is scientific acceptance criteria?
Checks that scientific code produces valid results, not merely that it executes. Domain assumptions and test coverage still determine what counts as a trustworthy success.
Why does this matter?
Author-reported preprint; no discussion sampled.
- Significance: executable feedback helps train research agents.
- Significance: verified trajectories are only as good as their criteria.
- Significance: transfer to new domains needs external replication.