What's this about?
Spatial-Interactor trains models on observation-action-observation sequences so they learn how objects and viewpoints change, then integrate those transitions across long interactions.
What exactly is on-policy distillation?
The student generates its own reasoning trajectory, while a privileged teacher with extra transition descriptions supplies guidance. Training therefore corrects the states the student actually visits.
Why does this matter?
author-reported preprint; 44 HF upvotes.
- LSI-108K mixes simulated and real interaction trajectories.
- The curriculum separates passive world changes, self-motion and long-horizon integration.
- Reported gains span several VLMs, but the abstract does not establish robustness in uncontrolled physical settings.