What's this about?
A new paper on RLMs argues transformers inherently struggle on tasks they weren't trained on — and that's fine: the *harness* (the scaffolding of loops, tools, and composition around the model) should be what generalizes, by composing trained skills into unseen combinations (2.5k likes).
What exactly is a "harness"?
Everything wrapped around the raw model: the loop that lets it retry, call tools, split tasks, check work. Model = engine; harness = the rest of the car. The paper's claim: stop expecting the engine to drive anywhere on its own — build drivetrains that recombine what it can already do.
What's the community saying?
- swyx spotlights a buried section: labs can "goalseek" benchmarks by training on test lookalikes without literally training on the test — an open secret now in print.
- That lands on the same day as the Fireworks routing study and Cursor's 15x cost spread — a converging "architecture over raw model" narrative.
- Links to yesterday's FluxBench finding that a purpose-built harness beat Claude Code 8.4x on chip design.