Tag
RecVerse is a GUI-grounded simulation agent that uses hierarchical memory and trajectory-level reinforcement learning to simulate realistic multi-turn e-commerce shopping sessions, addressing memory and optimization challenges in existing simulators.
UI-Mate-27B is an open-weight foundation GUI agent from Tencent that performs long-horizon computer tasks using live screenshots and demonstration-guided adaptation.
Introduces KnowAct-GUIClaw, a framework for personal GUI assistants with self-evolving memory and skill, achieving state-of-the-art performance on the MobileWorld benchmark and outperforming closed-source models like GPT-5.5.
This paper introduces UI-MOPD, a method combining multi-teacher on-policy distillation with continual learning for training GUI agents across multiple platforms, along with the Uni-GUI dataset. It achieves task success rates of 38.2% and 12.0% on OSWorld and MobileWorld respectively, demonstrating effective cross-platform capability retention and adaptation.
This technical report presents Xiaomi-GUI-0, a native multimodal GUI agent trained and evaluated in real-device environments using a hybrid infrastructure and a progressive training pipeline, achieving high success rates and improved stability on real-world mobile tasks.
GUICrafter introduces a weakly-supervised GUI agent that leverages massive unannotated screenshots and a two-stage curriculum learning framework to reduce reliance on expensive human annotations, achieving competitive performance with advanced systems like UI-TARS using only 0.1% of its data.
A self-improving GUI agent that learns from its mistakes, using a perception stack with YOLO and OCR, calling LLM only when needed, and running locally. It's open source under the MIT license.
H Company released Holo 3.1, an open-source computer-use LLM specialized for local deployment, achieving 79.3% on AndroidWorld benchmark, beating larger models like Qwen3.5-397B and Kimi-K2.5.
ByteDance open-sourced the UI-TARS-desktop project, containing a general multimodal Agent framework Agent TARS and a local GUI Agent UI-TARS Desktop, supporting real-world tasks in terminal/browser, based on UI-TARS vision model and Seed-1.5-VL, under Apache 2.0 license.
This paper introduces PAGER, a topology-aware agent that bridges the semantic-execution gap in point-precise GUI control, achieving 4.1x higher task success than baselines on the new PAGE Bench.
UI-TARS-desktop is a highly popular open-source tool by ByteDance that enables 100% local multimodal desktop automation, allowing users to control apps and browsers via natural language without cloud data leaks.
OmniGUI introduces a step-level benchmark for GUI agents that integrates static images, synchronous audio, and video clips to simulate real smartphone interactions. Evaluation shows current models struggle with temporal and auditory inputs, highlighting the need for omni-modal capabilities.
UI-TARS-2 is a native GUI-centered agent model that addresses data scalability, multi-turn RL, and environment stability challenges, achieving state-of-the-art results on GUI benchmarks (88.2 on Online-Mind2Web, 47.5 on OSWorld, 50.6 on WindowsAgentArena,73.3 on AndroidWorld) and outperforming Claude and OpenAI agents.
Page Agent is an open-source JavaScript library from Alibaba that lets you control web interfaces with natural language, without requiring browser extensions or headless browsers.
OpenAI demonstrates the 'Computer Use' feature in Codex, allowing the AI to directly interact with local GUI applications on macOS using an accessibility framework and the fast Spark model for non-blocking, high-speed automation.
ByteDance released TARS, a multimodal AI agent stack comprising Agent TARS (a CLI/Web UI-based general AI agent for GUI, browser, and terminal tasks) and UI-TARS Desktop (a native desktop application powered by the UI-TARS model for local and remote computer/browser automation). The stack integrates multimodal LLMs with MCP tools for human-like task completion.