What's this about?
OpenAI and Ironclad converted eleven real contracting and procurement workflows into detailed training and evaluation environments for computer-use models.
What exactly is criteria-based evaluation?
Instead of marking a task simply pass or fail, evaluators check many required details—approval thresholds, clauses and exception paths—to see where a workflow breaks.
Why does this matter?
Astra improved the internal score and simulated time but still averaged only 55%, showing why professional workflows need domain-built environments and human review. Different reasoning settings also limit a clean model-to-model comparison.