Tag
SoftwareGym提出了一个自动化流水线,用于合成多样化的GUI软件环境以训练计算机使用代理(CUAs),合成182个应用并构建了SoftwareGym-Bench评测套件,在OSWorld基准上将Pass@3从41.39%提升至56.97%。
CUAWright introduces a minimal ~3K-line terminal harness that uses bash commands and a filesystem as the sole action interface for digital agents, outperforming GUI-native harnesses on OSWorld 2.0, Online-Mind2Web, Odysseys, and CAD benchmarks while reducing estimated cost.
SimTrace is an open-source framework that generates faithful, fine-grained synthetic multimodal user clickstreams using a computer-use agent grounded in anonymized real trajectories and simulated web environments. It outperforms baselines on 7 of 8 fidelity metrics and improves next-action prediction by 11.0% when augmenting real data.
DeskForge introduces a controllable desktop environment that composes real applications to generate 1.2M densely annotated desktop observations (DeskForge-1M) for training computer-use agents. Fine-tuning four VLMs on this data yields significant gains on GUI grounding benchmarks (e.g., Qwen3.5-4B improves 11.51 points on ScreenSpot-Pro) and long-horizon task completion.
SCOUT introduces a two-stage agentic safety verifier that combines reasoning-intensive rubric generation with tool-intensive evidence gathering to detect unsafe behaviors in computer-use agents, improving unsafe detection on AutoElicit-Bench and reducing unsafe execution rates from 30.2% to 17.2% via test-time reflection.
OSWorld-Science introduces a benchmark and evaluation environment for computer-using VLM agents on scientific software, covering 146 expert-designed tasks across domains like molecular drawing, pathology imaging, statistics, and physical simulation with artifact-based evaluators and a special agent harness.
The paper proposes HybridCUA, a method that combines GUI and CLI interactions for computer-use agents, using a two-stage training framework with supervised fine-tuning and reinforcement learning to improve task accuracy and cross-platform generalizability.
The paper introduces KNOWS, a benchmark for evaluating web agents on complex, long-horizon tasks that involve synthesizing and organizing knowledge into artifacts, revealing that current agents struggle with visual steps and long-horizon reasoning.
CUA-SWE introduces a benchmark, environment, and evaluation pipeline that unifies computer-use agents and coding agents, testing whether frontier agents can complete software engineering tasks requiring visual feedback from running applications, across four SWE domains with deterministic tests.
Cognitive Sharding separates cognitive functions across specialist models to enable computer-use agents to run locally on 16 GB of memory, prioritizing accuracy and error containment over speed.
This paper introduces CAVEAT, a benchmark for evaluating computer-use agents in incentive-misaligned environments, revealing that agents often fail to preserve user objectives and proposes interventions to improve robustness.
This paper introduces SCOPE, a joint training method for computer-use agents that improves both task completion and safety, using a synthesized dataset and achieving strong performance on benchmarks OSWorld and OS-BLIND.
OSWorld-Pro introduces a process-based benchmark with over 300 tasks and 2,800 subgoals, grounded in 67,000+ human annotations, to evaluate computer-use agents step-by-step rather than only by end-state outcomes. It reveals failure modes even in state-of-the-art models, with top performers like Claude Opus 5 scoring 75.7% versus 83.4% on OSWorld.
RecreationWorld introduces a scalable and verifiable framework for hybrid computer-use agents, providing environments across five platforms and a benchmark for evaluation.
ERPBench introduces a benchmark for evaluating screenshot-only computer-use agents on a live ERP system, showing that strong general GUI performance does not transfer to enterprise reliability and includes a production harness with human approval for safe deployment.
HazardAuditor introduces an execution-grounded framework for supervising safety in computer-use agents, with Guard Policy Optimization improving safety outcomes by up to 16.5% accuracy over prior methods.
The paper introduces GUI-Primitives, a benchmark of 994 contrastive instruction pairs to diagnose spatial reasoning failures in vision-language models for GUI grounding, revealing that most failures stem from candidate localization rather than relation understanding.
Task Model Induction (TMI) is a method that induces structured task models from raw computer-use traces, improving agent performance by 30% and enabling auditable, reusable skills for real-world workflows.
ComponentBench introduces a benchmark and diagnostic pipeline for evaluating computer-use agents on component-level interactions in modern web UIs, addressing gaps in current evaluation methods by focusing on realistic, short interactions to diagnose failures across models.
This paper presents a deterministic, zero-model pipeline that compiles passively captured screen activity into structured 'activity frames' for agent memory, reducing a day of raw capture to a prompt-ready context block 86× smaller and achieving 98.4% QA accuracy. It also introduces measurements of routine overhead ratio and recurrence to model agent costs.