Tag
The article details the virtual machine technologies powering mobile AI agents, such as Claude Code's Firecracker microVM and Instinct's use of E2B sandboxes, highlighting their security and lifecycle management.
The tweet discusses how functional agents on mobile could drive voice AI growth, with the author noting increased voice dictate use due to Instinct, and references Jane Manchun Wong's tweet about Instinct's 'Talk to Instinct' feature for iPhone.
The author questions traditional mobile agent approaches relying on API access and proposes using screen understanding via hardware like aiden-firmware to enable more general-purpose agents that interact like humans.
The paper introduces CRATE, a two-stage framework using step-level consequence reasoning to evaluate mobile agents, achieving high F1-scores on benchmarks like AndroidWorld and MobileRisk.
CoAdapt-GUI is a test-time adaptation framework for mobile GUI agents that jointly adapts workflow context and policy, improving performance on unseen-app benchmarks like AndroidWorld-Generalization and AndroidWorld Plus.
This paper introduces MobileJudgeBench, a benchmark with 931 human-annotated trajectories for systematically evaluating LLM-based judges on mobile agent tasks. It finds that simple baseline judges with sampled screenshots rival purpose-built methods, with the LLM backbone being the primary driver of quality.
Introduces AndroidReality, a perturbation-based framework for evaluating and improving the robustness of mobile agents, with a taxonomy of real-world interface perturbations and a training-free Test-Time Introspective Recovery (TTIR) mechanism.
Introduces iOSWorld, an interactive native iOS simulator benchmark with persistent user identity across 26 apps, designed to evaluate personalized mobile agent capabilities through 133 tasks of increasing difficulty.
MIRAGE is a framework for mobile GUI agents that replaces verbose chain-of-thought reasoning with compact continuous latent representations, incorporating a generative world model perspective to predict future screen states before acting. On AndroidWorld and AndroidControl benchmarks, it achieves competitive or superior performance while reducing generated tokens by over 75%.
This paper proposes a Pre-Reasoning Perception Framework (PRPF) for proactive mobile agents, decoupling intervention timing from assistance generation to improve efficiency and reduce false triggers.
The author observes that the hardest part of phone-use AI agents is tracking state changes, as mobile interfaces have more dynamic and interruptive UI changes compared to desktop, and asks for others' experience.