gui-agents

Introduces MacArena, a benchmark of 421 tasks across 50 applications for evaluating computer-use agents on macOS, highlighting that existing benchmarks may not capture macOS-specific challenges.

StainFlow introduces an entity-stain-flow process reward model for GUI agents, using global entity stain tracking and local evidence linking to improve credit assignment in reinforcement learning, achieving 3.2% relative improvement on AndroidWorld.

DeskCraft is a new benchmark for evaluating desktop GUI agents on long-horizon professional creative workflows, incorporating human-in-the-loop collaboration protocols. It tests agents on tasks requiring over 50 steps across design, video, audio, and 3D software.

Introduces GUI-RobustEval, a benchmark for error recovery in GUI agents, and Robustness-driven Trajectory Synthesis (RoTS) to generate training data, achieving state-of-the-art on OSWorld.

AQuaUI is a training-free inference-time token reduction method for GUI agent models that uses adaptive quadtrees to reduce spatial redundancy in screenshots, achieving up to 13.22% speedup and 29.52% fewer visual tokens while retaining 99.06% of performance.

MementoGUI introduces a plug-in agentic memory framework for GUI agents that uses learned controllers for selective memory management and retrieval, improving performance on long-horizon tasks with compressed visual and textual representations.

Proposes Video2GUI, a framework to automatically extract GUI interaction trajectories from unlabeled instructional videos, building WildGUI dataset with 12M trajectories across 1500+ apps. Pre-training on this data yields 5-20% improvements on GUI grounding and action benchmarks.

WebHarbor packages 15 real websites (Amazon, GitHub, BBC, etc.) as self-contained Flask+SQLite apps in a single Docker image with sub-second reset, designed for reproducible web agent evaluation and training. The project invites community contributions to expand to 100+ sites, with co-authorship opportunities.

ClawGUI is an open-source framework for training, evaluating, and deploying GUI agents using reinforcement learning, featuring standardized benchmarks and cross-platform deployment to Android, iOS, and HarmonyOS.

The MAI-UI technical report presents a family of foundation GUI agents in multiple sizes, addressing real-world deployment challenges with a self-evolving data pipeline, device-cloud collaboration, and online RL, achieving state-of-the-art results on GUI grounding and mobile navigation benchmarks.