What's this about?
Large vision-language models have advanced embodied navigation considerably, but deploying them directly remains awkward. This paper (44 upvotes) argues the reason is that existing methods force VLMs into unnatural action spaces that misalign with how the models actually represent the world, and proposes an approach built around pointing, thinking, memorizing and aligning.
What exactly is an "action space" mismatch?
A robot's controller expects specific commands: rotate 15 degrees, advance 0.5 metres. A vision-language model reasons in language and imagery — "the doorway is on the left, behind the chair." Making the model emit controller commands forces it to translate from its natural representation into precise numbers, which is where errors enter. Aligning the action space to the model means letting it express intent in terms it handles well — pointing at a location in the image, for instance — and having the system convert that into motion. The analogy is asking someone for directions: pointing is reliable, reciting bearings in degrees is not.
Why is this worth noticing?
- It's the same insight as the harness literature in a different domain: don't change the model, change the interface it acts through.
- Navigation is the foundation for essentially all embodied work — a robot that can't reliably get to the right place can't do anything once there.