Splitting planning from clicking made my desktop agent much faster: frontier model plans, small model acts

Reddit r/AI_Agents Tools

Summary

作者分享了将桌面智能体的规划与执行分离的架构:由 Claude Code 等前沿模型批量规划步骤,小型快速模型 Jev 负责执行点击与操作,普通步骤跳过模型,从而显著提升速度与降低成本,测试中 12 个随机按钮达成 47/48、约 1.2 秒每次点击。代码已开源。

I've been building a desktop agent for my Mac, and the biggest speedup came from splitting planning from acting. The problem: when a frontier model drives a GUI directly, every click costs a screenshot plus a full reasoning pass. That's slow (several seconds a step) and expensive, even though most steps are trivial. What I do now: - Claude Code (the planner) writes the whole batch of steps up front: "click Upload Document, then type the name, then choose Resume from the dropdown". - A small, fast decision model (Jev, from TypeSafe) executes each step: it finds the target in a screenshot in about 0.6s and picks the move. - Plain steps (type text, press a key, choose from an open dropdown, attach a file) skip the model completely. - It reads the macOS accessibility tree first, and only falls back to pixels when an app exposes nothing. - After each batch it hands the planner a ~20-line text report of the screen, so the planner rarely needs a screenshot. Guardrails: it refuses clicks that look like send/delete/pay unless I've approved them, and it stops the moment I touch the mouse. On a test page of 12 randomly placed buttons it hit 47/48, at about 1.2s per click. It still gets lost in apps with custom-drawn UI (GitHub's web editor ate my keystrokes today). Curious whether others have landed on the same planner/executor split, or found a better way to handle apps with no accessibility tree. Code is open source; link in the comments per the sub rules.
Original Article

Similar Articles

@GoSailGlobal: Practical data on multi-agent AI collaboration: Use Opus 4.8 for planning, Deepseek/Gemma for execution — 10x cost reduction, 2x speed improvement. The secret is not using the most expensive model, but having cheap models do the heavy lifting and expensive models only make decisions. This is the same as company management: the CEO shouldn't write code, and interns shouldn't set strategy. A…

X AI KOLs Timeline

A practical sharing on multi-agent AI collaboration, proposing a hierarchical strategy using Opus 4.8 for planning and Deepseek/Gemma for execution, achieving a 10x cost reduction and 2x speed improvement, with open-source implementation.

@AI_Community_9: Unsloth Desktop takes the experience of Qwen3.8 27B to new heights. Out-of-the-box + cross-platform, the core is optimizing VRAM for fine-tuning and inference to the extreme (VRAM saved by 70%, speed increased by 2 times). It packs 'training, inference, and Agent toolchain' into a free, open-source desktop application, …

X AI KOLs Timeline

Unsloth Desktop is a free, open-source desktop application that integrates training, inference, and Agent toolchain through extreme optimization of VRAM and speed, significantly improving the user experience of AI models like Qwen3.8 27B.

When working on agent workflows or autonomous agent loops, a major bottleneck affecting experience is: every small branch decision, tool routing, task completion check, or safety interception, if all handled by expensive reasoning LLMs, not only brings seconds of delay per call, but token costs also...

X AI KOLs Timeline

This article introduces the open-source list awesome-jev-projects, which decouples System 1 fast decision models from System 2 reasoning LLMs to optimize AI agent workflows, reduce latency and costs, and provide code-level references.

@yibie: Recommend this project. shadcn (creator of shadcn/ui) built an Agent Skill — let the most expensive model handle auditing and planning, and let the cheap model write code. The idea itself isn't new, but shadcn turned it into an installable, orchestrated system with an execution closed loop. Every design decision is practical...

X AI KOLs Timeline

shadcn released the Agent Skill project improve, which has high-cost models perform code auditing and planning while low-cost models execute, forming an installable, orchestrated system with an execution closed loop.