Paper page - CUA-Gym: Scaling Verifiable Training Environments and Tasks for Computer-Use Agents

Source: https://huggingface.co/papers/2605.25624 Authors:

,

,

,

,

,

,

,

,

,

,

,

,

Abstract

RLVR framework for computer-use agents addresses data scarcity through scalable generation pipeline and synthetic environments, achieving superior performance on verification and transfer benchmarks.

Reinforcement learning with verifiable rewards(RLVR) has driven breakthroughs in domains such as math, tool-use, and software engineering, yet its extension tocomputer-use agents(CUAs) has been bottlenecked by the scarcity of scalable training data with deterministic rewards. Constructing such data for CUAs requires consistenttask instruction,executable environment, andverifiable reward. However, hand-curated benchmarks achieve high reward fidelity but cover few applications and LLM-as-judge-based datasets scale broadly but lack reliable verification. We present CUA-Gym, a scalable pipeline that co-generatestask instructions, environment states, and reward functions. Concretely, aGenerator agentconstructs the initial and golden environment states, and a separateDiscriminator agentwrites the reward function from the task specification. Anorchestrator agentdrives the two through iterative rounds upon execution. Generated tuples then pass a final filter combiningLLM majority votingandagent rollouts, ensuring quality beyond the per-task adversarial loop. To address the scarcity of training environments, we further synthesizeCUA-Gym-Hub, a broad suite of high-fidelity mock web applications grounded in real-world software-use distributions, expanding the scale of CUA RLVR data by magnitude. Using this pipeline, we construct CUA-Gym, a dataset of 32,112 verified RLVR training tuples grounded in 110 environments. Trained withGSPOon CUA-Gym, our CUA-Gym-A3B and CUA-Gym-A17B achieve 62.1% and 72.6% onOSWorld-Verified, outperforming prior open-source CUAs at comparable scales, with performance scaling smoothly in both data volume and environment diversity. The same checkpoints also improve on the held-outWebArenabenchmark, indicating transfer beyond the training environments. We will open-source the full synthesis pipeline, dataset,CUA-Gym-Hubenvironments, and models.

View arXiv pageView PDFProject pageGitHub8Add to collection

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.25624 in a model README.md to link it from this page.

Datasets citing this paper1

#### xlangai/CUA-Gym Viewer• Updatedabout 7 hours ago • 7.9k • 145

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.25624 in a Space README.md to link it from this page.

Collections including this paper1