computer-use

Matt Shumer speculates that combining Claude Opus 5.5 with Codex-level computer use would be unbeatable, expressing enthusiasm for future AI model capabilities.

Sonny is a local-first AI assistant for Windows that can listen via microphone, see via camera, browse the web, control the computer, maintain memory, and speak responses, with the core AI stack running entirely on-device.

LCU is an open-source tool that exposes Codex's original computer-use runtime outside the Codex app, letting agents in Pi, Claude Code, and other harnesses control the entire desktop with native app-approval panes and Touch ID-based permission management.

OpenAI's GPT-6 Astra model cleared the Orc starting zone in World of Warcraft in 40 minutes with zero deaths, using the open-source agent-wow client to navigate 'blind' by parsing raw server network packets and SQL quest data instead of viewing rendered frames.

A user discovers that Codex's computer-use capability can be exposed as a local MCP server and called from other harnesses like Claude Code, reporting faster and more accurate performance than the cua-driver alternative.

The team behind Agent S (ICLR 2025) opened Sai, a computer-use API that compiles plans into self-healing code instead of re-reasoning from screenshots, cutting tokens by up to 90% on repeat tasks. Sai tops OSWorld 2.0 with a 73% score at $15.70 per task, beating Claude Opus 5 (70.6% at $23.70) and GPT-5.6 Sol (62.6% at $26.62).

A practitioner questions whether computer-use AI agents are an inefficient waste of tokens, asking about the underlying economics of agentic browser/desktop automation.

Ant Group showcases Ling-3.1-flash powering planning inside its internal Tiger Agent, demonstrating multi-tool workflows like GitHub trending digests, CSV risk reports, and GLSL courseware, with early public access via Novita on Vercel AI Gateway (256K context) and open source planned.

autotrust released GEV-26B-Decide, a 26B-parameter (~4B active per token) decision model with adaptive thinking built on Gemma-4-26B-A4B-it, scoring 62.48 on the Decision Index. It also demos sub-100ms System 1 decisions for computer use (95% task success in headless Chromium) and MuJoCo robot-arm pick-and-place.

Higgsfield AI has integrated Computer Use into its platform through a new Codex/ChatGPT extension powered by GPT-6.1 Sol, letting the agent operate the full Higgsfield interface to chain ideation, image/video generation, and editing workflows end-to-end—reportedly enabling a demo agent to generate dish photos for coffee shops and pitch them via email for $450 in revenue.

The paper introduces AutoGUIWorld, a data-generation framework that uses image generators as visual world models to synthesize GUI interaction trajectories without running software, yielding 79,266 annotated samples that fine-tuning Qwen3.5-35B-A3B to improve OSWorld task score from 33.0% to 40.8% and ScienceBoard success rate from 14.0% to 32.2%.

Reduck MCP introduces a memory layer for browser agents that lets them write, store and reuse navigation scripts executed on the user's logged-in Chrome, making web browsing tasks 2-3x faster and cheaper than screenshot-driven Computer Use. A curated library of official scripts for sites like LinkedIn, Google Maps, Instagram and Stripe helps solve the cold-start problem.

WinMind is an early-stage Windows MCP server that uses the accessibility tree for agent interactions instead of screenshots, aiming to improve efficiency and reliability in computer-use agents.

autotrust releases JEV-27B-VL, a vision-enabled version of the JEV-27B model built on Qwen3.8-27B that provides fast calibrated System 1 decisions (via POST /v1/decide) alongside full System 2 reasoning, demonstrated on robot-arm pick-and-place (75% success) and computer-use browser tasks (95% completion).

OpenAI announced at DevDay a new capability allowing AI agents to be given their own computer and goals for autonomous work, marking a shift from chat-based interaction to task assignment.

cua released a new version of its open-source computer-use framework, letting AI click and type on desktops like a human so developers can build agents without reinventing the wheel — directly challenging proprietary offerings like Manus.

Incredible is an AI tool for Mac and PC that performs clicking and typing tasks on your behalf, operating in your signed-in browser with a plan preview, confirmation prompts before actions, and scheduled recurring tasks. It is now launching on Product Hunt as a 14-day free trial.

H Company has released two vision-language models, Holo4-27B-GGUF and Holo4-35B-A3B-GGUF, designed for computer use automation. These models, built on Qwen architectures, work with the hai-agents harness to execute tasks like clicks, typing, and code.

H company has released Holo4, a family of generalist AI models capable of performing computer-use tasks like clicking, coding, and calling tools across desktop, web, Android, and APIs, available commercially via the H Models API.

Hcompany releases Holo4, a series of agentic AI models for generalist computer-use, available in 27B and 35B-A3B sizes, which perform well on benchmarks compared to frontier models.