What's this about?
Agent harnesses have substantially improved performance on long-horizon tasks by coordinating how the agent interacts with its environment. But reinforcement learning *through* a complex harness is largely unexplored, because scaling that training is difficult. This paper (12 upvotes) explores doing it black-box.
What exactly does "black-box RL through a harness" mean?
Reinforcement learning normally needs to know how a change to the model affects the outcome, so it can push in the right direction. But a harness sits between the model and the result — tools, retries, control flow, summarization — and it isn't differentiable; you can't trace the gradient back through it. Black-box means treating everything between action and reward as an opaque function: try things, observe what scores well, adjust accordingly, without needing to understand the mechanism. It's slower and noisier than gradient-based learning, but it works on systems you can only run rather than analyze.
Why is this worth noticing?
- It's the third distinct attack on the harness in two weeks — DarwinX evolves it, OpenART showed it dominates safety outcomes, and this trains through it.
- It targets the actual deployed object. Almost all training optimizes the model in isolation, while what users run is model-plus-harness, and the gap between those two is where deployment surprises live.