computer-use-agents

Tag

Cards List
#computer-use-agents

SoftwareGym: Scaling Computer-Use Agent Training with Diverse Synthesized Software Environments

Hugging Face Daily Papers ↗ · 2026-10-08 Cached

SoftwareGym提出了一个自动化流水线,用于合成多样化的GUI软件环境以训练计算机使用代理(CUAs),合成182个应用并构建了SoftwareGym-Bench评测套件,在OSWorld基准上将Pass@3从41.39%提升至56.97%。

0 favorites 0 likes
#computer-use-agents

CUAWright: A Minimal Unified Interface for Digital Agents

Hugging Face Daily Papers ↗ · 2026-10-02 Cached

CUAWright introduces a minimal ~3K-line terminal harness that uses bash commands and a filesystem as the sole action interface for digital agents, outperforming GUI-native harnesses on OSWorld 2.0, Online-Mind2Web, Odysseys, and CAD benchmarks while reducing estimated cost.

0 favorites 0 likes
#computer-use-agents

SimTrace: Grounded Multimodal User Trajectories Generation for Online User Modeling

arXiv cs.AI ↗ · 2026-10-01 Cached

SimTrace is an open-source framework that generates faithful, fine-grained synthetic multimodal user clickstreams using a computer-use agent grounded in anonymized real trajectories and simulated web environments. It outperforms baselines on 7 of 8 fidelity metrics and improves next-action prediction by 11.0% when augmenting real data.

0 favorites 0 likes
#computer-use-agents

DeskForge: Dense Supervision from Desktop Environments for Computer-Use Agents

Hugging Face Daily Papers ↗ · 2026-10-01 Cached

DeskForge introduces a controllable desktop environment that composes real applications to generate 1.2M densely annotated desktop observations (DeskForge-1M) for training computer-use agents. Fine-tuning four VLMs on this data yields significant gains on GUI grounding benchmarks (e.g., Qwen3.5-4B improves 11.51 points on ScreenSpot-Pro) and long-horizon task completion.

0 favorites 0 likes
#computer-use-agents

SCOUT: Synergizing Reasoning and Tool-Use for Computer-Use Safety

arXiv cs.CL ↗ · 2026-09-30 Cached

SCOUT introduces a two-stage agentic safety verifier that combines reasoning-intensive rubric generation with tool-intensive evidence gathering to detect unsafe behaviors in computer-use agents, improving unsafe detection on AutoElicit-Bench and reducing unsafe execution rates from 30.2% to 17.2% via test-time reflection.

0 favorites 0 likes
#computer-use-agents

OSWorld-Science: A Benchmark of Computer Use Agents for Learning and Using Scientific Software

Hugging Face Daily Papers ↗ · 2026-09-30 Cached

OSWorld-Science introduces a benchmark and evaluation environment for computer-using VLM agents on scientific software, covering 146 expert-designed tasks across domains like molecular drawing, pathology imaging, statistics, and physical simulation with artifact-based evaluators and a special agent harness.

0 favorites 0 likes
#computer-use-agents

HybridCUA: Learning to Orchestrate GUI and CLI for Computer-Use Agents

Hugging Face Daily Papers ↗ · 2026-09-29 Cached

The paper proposes HybridCUA, a method that combines GUI and CLI interactions for computer-use agents, using a two-stage training framework with supervised fine-tuning and reinforcement learning to improve task accuracy and cross-platform generalizability.

0 favorites 0 likes
#computer-use-agents

The Hard Part Comes After Search: Benchmarking Web Agents on Synthesizing, Organizing, and Displaying Knowledge

arXiv cs.CL ↗ · 2026-09-28 Cached

The paper introduces KNOWS, a benchmark for evaluating web agents on complex, long-horizon tasks that involve synthesizing and organizing knowledge into artifacts, revealing that current agents struggle with visual steps and long-horizon reasoning.

0 favorites 0 likes
#computer-use-agents

CUA-SWE: When Computer-Use Agents Meet Visual Software Engineering

Hugging Face Daily Papers ↗ · 2026-09-26 Cached

CUA-SWE introduces a benchmark, environment, and evaluation pipeline that unifies computer-use agents and coding agents, testing whether frontier agents can complete software engineering tasks requiring visual feedback from running applications, across four SWE domains with deterministic tests.

0 favorites 0 likes
#computer-use-agents

Cognitive Sharding: Computer-Use Agents Can Run Locally on 16 GB

Reddit r/ArtificialInteligence ↗ · 2026-09-25

Cognitive Sharding separates cognitive functions across specialist models to enable computer-use agents to run locally on 16 GB of memory, prioritizing accuracy and error containment over speed.

0 favorites 0 likes
#computer-use-agents

CAVEAT: Towards Robust Computer-Use Agents in Incentive-Misaligned Environments

arXiv cs.AI ↗ · 2026-09-24 Cached

This paper introduces CAVEAT, a benchmark for evaluating computer-use agents in incentive-misaligned environments, revealing that agents often fail to preserve user objectives and proposes interventions to improve robustness.

0 favorites 0 likes
#computer-use-agents

Beyond Task Completion: Training Capable and Safe Computer-Use Agents

arXiv cs.LG ↗ · 2026-09-22 Cached

This paper introduces SCOPE, a joint training method for computer-use agents that improves both task completion and safety, using a synthesized dataset and achieving strong performance on benchmarks OSWorld and OS-BLIND.

0 favorites 0 likes
#computer-use-agents

OSWorld-Pro: Process-based Evaluation for Computer Use Agents

Hugging Face Daily Papers ↗ · 2026-09-21 Cached

OSWorld-Pro introduces a process-based benchmark with over 300 tasks and 2,800 subgoals, grounded in 67,000+ human annotations, to evaluate computer-use agents step-by-step rather than only by end-state outcomes. It reveals failure modes even in state-of-the-art models, with top performers like Claude Opus 5 scoring 75.7% versus 83.4% on OSWorld.

0 favorites 0 likes
#computer-use-agents

RecreationWorld: Scalable and Verifiable Environments for Hybrid Computer-Use Agents

Hugging Face Daily Papers ↗ · 2026-09-18 Cached

RecreationWorld introduces a scalable and verifiable framework for hybrid computer-use agents, providing environments across five platforms and a benchmark for evaluation.

0 favorites 0 likes
#computer-use-agents

ERPBench: A State-Grounded Evaluation Paradigm for Computer-Use Agents in Enterprise Software

arXiv cs.AI ↗ · 2026-09-17 Cached

ERPBench introduces a benchmark for evaluating screenshot-only computer-use agents on a live ERP system, showing that strong general GUI performance does not transfer to enterprise reliability and includes a production harness with human approval for safe deployment.

0 favorites 0 likes
#computer-use-agents

HazardAuditor: From Executable Threats to Safer Computer-Use Agents

Hugging Face Daily Papers ↗ · 2026-09-14 Cached

HazardAuditor introduces an execution-grounded framework for supervising safety in computer-use agents, with Guard Policy Optimization improving safety outcomes by up to 16.5% accuracy over prior methods.

0 favorites 0 likes
#computer-use-agents

GUI-Primitives: Diagnosing Spatial Reasoning Failures in Vision-Language GUI Grounding

arXiv cs.CL ↗ · 2026-08-25 Cached

The paper introduces GUI-Primitives, a benchmark of 994 contrastive instruction pairs to diagnose spatial reasoning failures in vision-language models for GUI grounding, revealing that most failures stem from candidate localization rather than relation understanding.

0 favorites 0 likes
#computer-use-agents

@dair_ai: This is one of the most effective ways to improve your agentic workflows. If you are building computer-use agents, this…

X AI KOLs Timeline ↗ · 2026-08-22 Cached

Task Model Induction (TMI) is a method that induces structured task models from raw computer-use traces, improving agent performance by 30% and enabling auditable, reusable skills for real-world workflows.

0 favorites 0 likes
#computer-use-agents

ComponentBench: Diagnosing Component-Level Failures in Computer-Use Agents

arXiv cs.AI ↗ · 2026-08-20 Cached

ComponentBench introduces a benchmark and diagnostic pipeline for evaluating computer-use agents on component-level interactions in modern web UIs, addressing gaps in current evaluation methods by focusing on realistic, short interactions to diagnose failures across models.

0 favorites 0 likes
#computer-use-agents

Activity Frames: Deterministic Screen-Activity Compilation for Agent Memory and Replay

arXiv cs.AI ↗ · 2026-08-07 Cached

This paper presents a deterministic, zero-model pipeline that compiles passively captured screen activity into structured 'activity frames' for agent memory, reducing a day of raw capture to a prompt-ready context block 86× smaller and achieving 98.4% QA accuracy. It also introduces measurements of routine overhead ratio and recurrence to model agent costs.

0 favorites 0 likes
← Previous
Next →
← Back to home

Submit Feedback