What's this about?
As on-device agents turn into personal assistants, the mobile operating system becomes the main testbed — and the authors argue existing benchmarks do not capture what real tasks demand. MobilePA-Bench is their attempt at complex real-world mobile tasks. 34 upvotes.
What exactly is a "planner agent"?
An agent that has to decompose a goal into steps before acting — "book me a table" becomes open the app, search, filter, check the calendar, choose, confirm. On a phone it is harder than on a desktop, because the agent works through a small touch interface with limited context and no command line.
What's the community saying?
The recurring theme in mobile-agent work is that benchmarks are the bottleneck: results are hard to reproduce because apps change under you, so a well-constructed benchmark is arguably more valuable than another agent.