Tag
Vercel Labs has open-sourced agent-desktop, enabling any coding agent to gain desktop-level Computer Use capabilities by directly reading the system accessibility tree rather than screenshots. It supports background operation and simultaneous control by multiple agents, and is compatible with macOS, Windows, and Linux.
作者分享了将桌面智能体的规划与执行分离的架构：由 Claude Code 等前沿模型批量规划步骤，小型快速模型 Jev 负责执行点击与操作，普通步骤跳过模型，从而显著提升速度与降低成本，测试中 12 个随机按钮达成 47/48、约 1.2 秒每次点击。代码已开源。
Work2Skill 提出一个框架，通过 WorkRecorder 收集人类 GUI 操作演示，构建 WorkDemo-350 多模态数据集，并用智能体式技能归纳方法从人类经验中提炼可复用的智能体技能，在 MyPCBench 和 OSWorld-Verified 上显著提升了多种执行模型的任务表现。
An AI agent developer argues that recovery rate after mistakes is more important than first-try success rate for practical usability, noting that no current benchmarks report recovery cases.
A developer shares four common accessibility tree pitfalls that break desktop agents: stale PIDs after app switches, modal sheets intercepting clicks, multi-monitor coordinate issues, and silent failures. Solutions include detecting frontmost app changes, explicit modal checks, and correct coordinate targeting.
Vercel Labs released agent-desktop, an open-source CLI and C library that lets AI agents read and operate macOS, Windows, and Linux desktop apps via the OS accessibility tree, using JSON snapshots with element refs instead of pixel matching.