What's this about?
Automatic rubric induction for research agents — deriving the evaluation criteria before trying to improve against them. 11 upvotes.
What is a rubric, and why induce one?
A rubric is the list of things a good answer must have, written before you grade. In research work there is no single correct answer, so improvement requires knowing what "better" means — and if a human has to write that rubric for every task, the whole loop stops scaling. Inducing it means having the system derive the criteria from the task itself. The risk is obvious and is the interesting part: a system that writes its own rubric can write one it happens to satisfy.
Why does this matter to you?
- It is the unverifiable-domain problem again — the same gap J-Zero attacks with a co-evolving judge.
- "Evaluate before improving" is the right order and the opposite of how most self-improvement loops are built.
- Self-authored evaluation criteria is precisely the failure mode the reward-hacking research demonstrated at scale this week.