What's this about?
Two multimodal papers at 16 upvotes each. Video-IFBench evaluates whether multimodal LLMs actually follow instructions in video-understanding scenarios. StreamPI does streaming multimodal temporal modelling for vision-language-action models.
What is a vision-language-action model?
A model that takes in pixels and text and outputs *actions* — motor commands, clicks, arm movements — rather than words. It is the architecture behind most current robot learning. "Streaming" matters because a robot cannot wait for a complete video clip before deciding what to do; it needs to act on partial, arriving-now input, which breaks most temporal modelling designed for whole clips.
Why does this matter to you?
- Instruction-following in video is underexamined — models are usually tested on description, not on doing what you asked.
- StreamPI is directly relevant to the embodied-AI thread running through today's digest via Microduck.
- Both are incremental, and both are the kind of work that decides whether the flashy demos generalise.