Tag
ZeroSphere is a product that provides AI with a virtual display on Windows, allowing it to interact with desktop applications while users maintain control over their own desktop.
Claude AI introduces a background computer use feature in Claude Cowork and Claude Code, enabling autonomous desktop task execution while the user focuses on other work.
Sai defeats Opus 5 and GPT-5.6 in the OSWorld 2.0 benchmark at roughly two-thirds the cost, scoring 73%, and demonstrates efficient desktop task execution capabilities.
This paper proposes Evidence-First Reflection (EFR) to improve reflection in desktop GUI agents by decoupling action-induced visual difference extraction from outcome verification, yielding accuracy gains of 7.11% on benchmarks.
The article introduces Windows Harness, an open-source Python tool designed for automating a wide range of Windows tasks, featuring UI automation, screenshots, raw input, and integration with browsers, files, and PowerShell.
pi-computer-use v0.4.3 is released, bringing parity with Codex, adding a ghost cursor, parallel tasks for subagents, and improved batching, resulting in 50% latency improvement and better handling of complex use cases like Microsoft apps.
A record-and-compile tool lets users demonstrate desktop tasks, then an LLM converts the session into structured skill files for agents to replay, aiming to reduce brittleness and lower the barrier to creating reusable agent capabilities.
Vantage is an open-source Windows app that uses an LLM to automate desktop tasks via natural language, enabling users to drive any desktop app with plain English commands.
China released an open-source desktop automation agent that runs 100% locally, capable of controlling desktop apps, files, and browsing without internet.
ByteDance open-sources UI-TARS Desktop, a 100% local desktop automation tool that operates purely on pixels with no API calls, resolving the two major pain points of data privacy and API costs, providing an efficient open-source solution for building private automation workflows.
Microsoft launches Fara-7B, an efficient Computer Use Agent with only 7B parameters, surpassing larger models on web tasks, supporting pure local deployment, and achieving low-cost desktop automation.
ProCUA-SFT is a large-scale synthetic dataset of 3.1M step-level SFT samples for training computer-use agents, produced via an automated pipeline using a single VLM (Kimi-K2.5). Fine-tuning UI-TARS 7B on it achieves 45.0% on OSWorld, an 18.7 point improvement over the base model.
Microsoft has released Fara-7B, a small 7B-parameter language model focused on pure local desktop automation. It can directly take over your mouse and keyboard to execute repetitive workflows, with low cost and no need for internet connectivity.
Minicor is a Y Combinator-backed platform that deploys self-healing AI agents for scalable desktop automations, enabling integration with legacy systems lacking APIs.
Midscene's Computer Agent enables desktop UI automation to run headless in Linux CI, automated via xvfb-run, without needing a real machine or VM, and supports Electron, Qt, and GTK applications.
Atomic-Agent is a desktop operation Agent designed for llama.cpp local inference models, optimizing the runtime architecture to enable small local models to reliably execute multi-step desktop tasks.
IrisGo, backed by Andrew Ng, launches an AI desktop companion that learns user workflows and automates repetitive tasks on-device for privacy, targeting knowledge workers.
The author discusses building a small VLM for desktop GUI automation to move data between apps without APIs, expressing interest in non-coding autonomous use cases for local models.
OpenComputer presents a framework for creating verifiable software environments for computer-use agents, integrating state verifiers, self-improving verification layers, task synthesis, and evaluation systems across 33 desktop applications. Experiments show its verifiers align better with human judgment than LLM-as-judge, and frontier agents struggle with end-to-end completion.
OpenAI is developing a feature for Codex to control macOS applications via Computer Use even when the laptop is locked or asleep, and to remotely operate other desktop devices running the Codex app, extending its remote control capabilities.