Tag
SoftwareGym提出了一个自动化流水线，用于合成多样化的GUI软件环境以训练计算机使用代理（CUAs），合成182个应用并构建了SoftwareGym-Bench评测套件，在OSWorld基准上将Pass@3从41.39%提升至56.97%。
OSWorld-Pro introduces a process-based benchmark with over 300 tasks and 2,800 subgoals, grounded in 67,000+ human annotations, to evaluate computer-use agents step-by-step rather than only by end-state outcomes. It reveals failure modes even in state-of-the-art models, with top performers like Claude Opus 5 scoring 75.7% versus 83.4% on OSWorld.
A preprint analyzing why computer-use agents succeed once but fail on repeated executions, attributing unreliability to execution stochasticity, task ambiguity, and behavioral variability, and advocating repeated evaluation and stable strategies.