MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
Summary
MobileGym is a browser-based simulation platform for mobile GUI agent research, featuring deterministic state evaluation and scalable parallel execution. It includes a benchmark of 416 tasks and demonstrates gains using GRPO on Qwen3-VL-4B.
View Cached Full Text
Cached at: 05/27/26, 02:47 AM
Paper page - MobileGym: A Verifiable and Highly Parallel Simulation Platform for Mobile GUI Agent Research
Source: https://huggingface.co/papers/2605.26114 Authors:
,
,
,
,
,
,
,
,
,
Abstract
MobileGym presents a browser-based mobile environment enabling deterministic evaluation and scalable reinforcement learning through JSON-based state management and parallel execution.
We present MobileGym, a browser-hosted, lightweight, fully controllable environment for everyday mobile use, targeting interaction fidelity without replicating proprietary backends. It enables two capabilities previously out of reach for everyday apps: verifiable outcome signals throughdeterministic state-based judgingoverstructured JSON state, and scalable online RL through low-costparallel rollouts. The full environment state is captured, configured, forked, and compared as structured JSON, and a single server can host hundreds of parallel instances, with about 400 MB memory per instance and about 3 s cold start. Alayered state modeland adeclarative task-definition frameworkkeep state programmability and task creation practical at scale, and a singleprogrammatic judging mechanismdelivers both deterministic evaluation verdicts and dense RL rewards. The accompanying MobileGym-Bench provides 416parameterized task templates, including 256 test and 160 train templates, over 28 apps, with deterministic judges and a structuredAnswerSheet protocolthat avoids free-text matching failures. In aSim-to-Real case study,GRPOonQwen3-VL-4B-Instructgains +12.8 percentage points on the 256-task test set, and on a 59-task real-device signal subset, real-device execution retains 95.1% of the simulation-side training gain. Project page: https://mobilegym.github.io.
View arXiv pageView PDFProject pageGitHubAdd to collection
Get this paper in your agent:
hf papers read 2605\.26114
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2605.26114 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2605.26114 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2605.26114 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
SimuWoB: Simulating Real-World Mobile Apps for Fast and Faithful GUI Agent Benchmarking
SimuWoB is a synthetic benchmark with 120 challenging tasks for mobile GUI agents, using high-fidelity virtual environments and automatic reward generation. Experiments reveal that current agents achieve only 27.92% average success rate, dropping to 17.82% on long-horizon tasks, indicating substantial weaknesses in complex scenarios.
ShopGym: An Integrated Framework for Realistic Simulation and Scalable Benchmarking of E-Commerce Web Agents
ShopGym is a framework that converts live e-commerce storefronts into self-contained sandbox shops for realistic, controllable, and reproducible benchmarking of web agents, with synthetic tasks across seven skill categories.
Workflow-GYM: Towards Long-Horizon Evaluation of Computer-use Agentic tasks in Real-World Professional Fields
Workflow-GYM is a benchmark for evaluating AI agents on long-horizon GUI tasks in professional domains. Experiments show that even top models achieve only ~30% success, revealing significant challenges.
MemGUI-Agent: An End-to-End Long-Horizon Mobile GUI Agent with Proactive Context Management
MemGUI-Agent introduces proactive context management for long-horizon mobile GUI tasks, using Context-as-Action (ConAct) to maintain critical information. It includes the MemGUI-3K dataset and achieves state-of-the-art performance on MemGUI-Bench and MobileWorld benchmarks with an 8B model.
CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents
CUA-Gym introduces a scalable pipeline for generating verifiable training environments and tasks for computer-use agents, addressing data scarcity. The resulting dataset and models achieve strong performance on benchmarks like OSWorld-Verified and WebArena.