What's this about?
Mid-training is increasingly recognized as a critical stage for shaping model capability, and recent work has shown targeted mid-training can strengthen reasoning. MidTool applies the same idea to agentic tool use, synthesizing mid-training data for it.
What exactly is mid-training?
It sits between the two stages everyone talks about. Pre-training is the enormous general run on web-scale text. Post-training is the comparatively small alignment and fine-tuning phase. Mid-training is a large-scale continued-training stage on curated data selected to build a specific capability — bigger and more foundational than fine-tuning, but far cheaper than starting over. The reason it's become interesting is that some capabilities appear to need to be built into the model's foundations rather than layered on top afterwards, and reasoning turned out to be one of them.
Why is this worth noticing?
- If tool use is like reasoning in this respect, it means the difference between models that use tools well and badly is set earlier than most people assume, and fine-tuning won't close it.
- It connects to Jason Wei's argument this week that internalized capability differs from bolted-on capability — mid-training is where internalization happens.