Efficient GUI Agents: A Systems Survey of Observation, Memory, Action, and Runtime Optimization

# A Systems Survey of Observation,Memory, Action, and Runtime Optimization
Source: [https://arxiv.org/html/2609.02309](https://arxiv.org/html/2609.02309)
Bizhe BaiAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaAffiliation:Shanghai Innovation Institute, Shanghai, ChinaJiakang YuanHongming WuAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaXinyue WangAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaJie RenAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaSiyao ChenAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaYuchen YaAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaFan BaiAffiliation:Independent ResearcherPai PengAffiliation:Independent ResearcherHuafeng QinAffiliation:Chongqing Technology and Business UniversityTao ChenAffiliation:College of Future Information Technology, Fudan University, Shanghai, ChinaAffiliation:Shanghai Innovation Institute, Shanghai, China

###### Abstract

GUI agents increasingly operate across websites, mobile apps, and desktop environments, yet the field still reports progress primarily through task success\. We argue that practical deployment depends equally on efficiency: how much context, computation, action budget, and runtime overhead an agent consumes while succeeding\. This survey studies efficient GUI agents through an end\-to\-end systems lens that preserves the current technical axes of observation efficiency, context and memory efficiency, action efficiency, and planner\-side/system efficiency\. For each subsection, we expand the seed literature through targeted search plus backward and forward citation chaining, then synthesize the dominant mechanisms, reported efficiency signals, and new overheads they introduce\. Across the literature, recent progress converges on a small set of recurring ideas: selective reading instead of full\-context ingestion, global\-to\-local visual allocation, recoverable memory rather than raw history replay, verification\-aware control, and hybrid runtimes that can switch between GUI and non\-GUI execution\. We conclude by identifying the main open problems, including honest accounting of verifier cost, cross\-benchmark comparability, and co\-design of observation, memory, and execution layers under real latency and privacy constraints\.

## 1Introduction

Large language models \(LLMs\) and vision\-language models \(VLMs\) have moved GUI automation toward open\-ended computer use: modern agents can interpret natural\-language instructions, inspect live graphical interfaces, and carry out multi\-step tasks across websites, mobile applications, and desktop operating systems[Yang et al\. \(2026b\)](https://arxiv.org/html/2609.02309#bib.bib76);[Sager et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib78);[Nguyen et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib77)\. This progress is reflected in interactive benchmarks such as Mind2Web, WebArena, VisualWebArena, BrowserGym, AndroidWorld, OSWorld, and Windows Agent Arena, which evaluate agents on web navigation, visually grounded web interaction, mobile\-app control, and open\-ended operating\-system tasks[Deng et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib69);[Zhou et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib70);[Koh et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib71);[Le Sellier De Chezelles et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib72);[Rawles et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib34);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib73);[Bonatti et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib74)\. Collectively, these benchmarks show that recent GUI agents are becoming capable of increasingly realistic computer\-use tasks, from navigating complex websites and operating mobile apps to completing multi\-step workflows in desktop environments\.

At the same time, these capabilities often come with substantial interaction and runtime costs\. A GUI agent may eventually complete a task, but only after many observation–reasoning–action cycles, repeated model calls, redundant interface operations, or long waits between steps\. OSWorld\-Human, for example, shows that leading computer\-use agents on OSWorld still take substantially more steps than human\-derived trajectories and that planning, judging, and reflection dominate end\-to\-end latency[Abhyankar et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib51)\. This matters in practice because GUI agents interact with the same visible interfaces that users depend on: an agent that occupies the screen for too long, performs unnecessary actions, or requires users to wait through slow intermediate steps can be difficult to deploy even when its final answer is correct\. Recent efficiency\-aware studies have therefore begun to profile step counts, visual\-token and cache costs, planning latency, action abstraction, and end\-to\-end runtime overhead in GUI\-agent execution[Huang et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib23);[Zhong et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib38);[Abhyankar et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib4)\. These findings expose a limitation of success\-only evaluation: an agent that eventually completes a task may still be too slow, too costly, too context\-heavy, or too interaction\-inefficient for practical deployment\.

Following recent surveys on efficient agents, computer\-use agents, and GUI agents[Yang et al\. \(2026b\)](https://arxiv.org/html/2609.02309#bib.bib76);[Sager et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib78);[Nguyen et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib77), we distinguish among three related notions:

LLM\.A foundation model trained on large\-scale language data, often used as the reasoning and natural\-language\-processing backbone of agentic systems\.Agent\.A goal\-directed system that repeatedly observes an environment, updates its internal state, and selects actions to make progress toward a task objective\.GUI agent\.A specific type of computer\-use agent whose primary perception and actuation channels are graphical user interfaces rather than text\-only APIs or command lines\.

In this survey, we use*agent*in the above closed\-loop sense\. A*GUI agent*is therefore not simply an LLM prompted with screenshots: it is a computer\-use agent whose observations and actions are tied to screenshots, DOM or HTML structures, accessibility trees, focused elements, window metadata, and interface\-level operations such as clicking, typing, scrolling, dragging, invoking shortcuts, or switching applications[Deng et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib69);[Zhou et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib70);[Rawles et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib34);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib73);[Nguyen et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib77);[Sager et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib78)\. Unlike text\-only tool agents, GUI agents must solve visual grounding, partial observability, long\-horizon state tracking, and execution under noisy or incomplete interface representations[Koh et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib71);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib73);[Nguyen et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib77);[Sager et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib78)\.Why is efficiency particularly critical for GUI agents?GUI interaction is intrinsically multimodal and observation\-intensive\. On the web, agents may need to parse long DOM or accessibility trees whose redundant structure inflates context length and reasoning cost[Abuelsaad et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib1);[Schiepanski and Piël \(2025\)](https://arxiv.org/html/2609.02309#bib.bib3);[Kerboua et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib6);[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib7)\. In screenshot\-centric settings, the agent must interpret visually dense screens, small targets, and weakly structured accessibility metadata, which makes grounding both computationally expensive and error\-prone[Cheng et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib8);[You et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib9);[Gou et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib17);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib73);[Bonatti et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib74)\. More broadly, literature on efficient multimodal\-LLM indicates that multimodal capability is often constrained by training cost, inference latency, memory pressure, and repeated visual computation[Jin et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib75)\. These issues become sharper in GUI environments because perception, reasoning, and action are interleaved at every step rather than amortized over a single query\. GUI tasks are also long\-horizon and error\-sensitive: agents must recover from failed actions, interface changes, and lengthy trajectories, and human profiling already shows large step and latency gaps in realistic computer\-use tasks[Agashe et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib15);[Abhyankar et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib51);[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67);[Rawles et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib34)\. Efficiency also shapes deployability and privacy, since practical systems may need to minimize exposed UI content, cloud traffic, and runtime overhead[Jin et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib75);[Fan et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib39);[Wang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib40)\.

This survey makes two contributions\. First, we organize the literature with an efficiency\-centered taxonomy spanning observation, context and memory, action, and planner\-side/system optimization\. Second, we synthesize the main cross\-layer trade\-offs and open challenges, including cases where apparent savings are offset by new parser, verifier, retriever, or orchestration costs\.

## 2Preliminaries

This section fixes the terminology and the system view used in the rest of the survey\. The goal is not to formalize every implementation detail, but to provide a compact description of what a GUI agent is, how it works, and where efficiency enters the loop\.

### 2\.1From Agents to GUI Agents

From the perspective of this survey, GUI agents are a specific class of agentic systems: they receive a user goal, observe a graphical interface, reason about the next step, and execute an interface\-level action[Yang et al\. \(2026b\)](https://arxiv.org/html/2609.02309#bib.bib76);[Sager et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib78)\. A compact way to write this loop is

at∼π⁡\(g,ot,mt\),a\_\{t\}\\sim\\pi\(g,o\_\{t\},m\_\{t\}\),\(1\)whereggis the user goal,oto\_\{t\}is the current GUI observation,mtm\_\{t\}is the retained context or memory, andata\_\{t\}is the next action\. We use this formulation only to fix notation\. In practice,oto\_\{t\}may include screenshots, DOM or HTML, accessibility trees, or related interface metadata, whileata\_\{t\}may include clicking, typing, scrolling, dragging, hotkeys, or app switching[Deng et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib69);[Zhou et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib70);[Rawles et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib34);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib73)\. A more detailed discussion of GUI\-agent is stated in Appendix[A](https://arxiv.org/html/2609.02309#A1)

### 2\.2How GUI Agents implement and What Should Be Efficient

Although implementations differ, most GUI agents follow the same functional pipeline\. They first construct an actionable representation of the current interface from screenshots, DOM or HTML, accessibility trees, or hybrid parsing outputs[Deng et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib69);[Zhou et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib70);[Cheng et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib8);[Gou et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib17)\. They then retain task\-relevant context through short\-horizon history or explicit memory structures, reason about the next subgoal or action, ground that decision to an executable element or coordinate, execute the action, and verify whether the intended effect occurred[Agashe et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib15);[Xie et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib73);[Lee et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib47);[Chen et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib60);[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67)\. Some systems collapse these stages into a single multimodal model, while others expose them as separate modules; the workflow remains the same\.

![Refer to caption](https://arxiv.org/html/2609.02309v1/GUI_figure1.png)Figure 1:A GUI\-agent system can be viewed as a loop of observation/perception, context and memory, planning/reasoning, grounding/action execution, and verification/feedback\. The four taxonomy axes of this survey are shown explicitly on the corresponding stages: Observation Efficiency, Context and Memory Efficiency, Action Efficiency, and Planner\-Side and System Efficiency\.This workflow also clarifies what this survey means by efficiency\.*Observation efficiency*concerns how the current interface is represented without overwhelming the model with redundant text or visual tokens[Abuelsaad et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib1);[Schiepanski and Piël \(2025\)](https://arxiv.org/html/2609.02309#bib.bib3);[Lin et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib42)\.*Context and memory efficiency*concerns how historical information is compressed, retained, or retrieved across steps[Liu et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib57);[Liu et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib21)\.*Action efficiency*concerns how economically the agent reaches the goal through abstraction, pruning, verification, and recovery[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib7);[Lee et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib47);[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67)\.*Planner\-side and system efficiency*concerns reasoning depth, orchestration overhead, backend routing, and end\-to\-end runtime beyond the next\-action predictor itself[Yang et al\. \(2026b\)](https://arxiv.org/html/2609.02309#bib.bib76);[Fan et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib39);[Wang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib40)\.

### 2\.3Efficiency Metrics for GUI Agents

Objective evaluation should separate*effectiveness*from*efficiency*\. Effectiveness metrics measure task quality; efficiency metrics measure the resources spent to obtain that quality\. We use the following vocabulary:

1. 1\.Latency metrics: TTFT is time from request dispatch to the first generated token; TPOT is average time per generated token after the first token; inference latency is total model\-call time; per\-step latency is one observe–plan–act–verify cycle; end\-to\-end latency is time from task start to success or failure\.
2. 2\.Observation metrics: perception calls count screenshot, crop, OCR, VLM, DOM, HTML, or AxTree parsing invocations; DOM/HTML/AxTree length measures serialized interface size; image crops count localized visual inputs\.
3. 3\.Context and memory metrics: history length, retrieved memory size, KV\-cache size, and memory footprint measure retained context, retrieved state, and serving memory cost\.

### 2\.4Organization of This Survey

The remainder of this survey is organized as follows\. Section 3 reviews Observation Efficiency\. Section 4 reviews Context and Memory Efficiency\. Section 5 reviews Action Efficiency\. Section 6 discusses Planner\-Side and System Efficiency\. Section 7 outlines Open Challenges and Future Directions\. For readability, the tree\-structured taxonomy is collected in Figure[2](https://arxiv.org/html/2609.02309#S2.F2), while the section\-wise paper summary tables are collected in the Appendix[D](https://arxiv.org/html/2609.02309#A4)\.

EfficientGUI AgentObservationEfficiencyContext & MemoryEfficiencyActionEfficiencyPlanner\-Side &System EfficiencyTextual Observation ReductionRegion\-Focused Visual PerceptionObservation Enrichment via Parsingand HybridizationSummary\-Based Compression andSelective Look\-BackRuntime\-Level RepresentationCompressionAction AbstractionAction Pruning, Verification,and RecoveryExploration ControlPlanner\-Side EfficiencySystem Efficiency Beyondthe PlannerAgent\-E \[[3](https://arxiv.org/html/2609.02309#bib.bib1)\];Beyond Pixels \[[49](https://arxiv.org/html/2609.02309#bib.bib3)\];LineRetriever \[[26](https://arxiv.org/html/2609.02309#bib.bib5)\];FocusAgent \[[25](https://arxiv.org/html/2609.02309#bib.bib6)\];Prune4Web \[[70](https://arxiv.org/html/2609.02309#bib.bib7)\];Read More, Think More \[[17](https://arxiv.org/html/2609.02309#bib.bib41)\]\.SeeClick \[[13](https://arxiv.org/html/2609.02309#bib.bib8)\];Ferret\-UI \[[68](https://arxiv.org/html/2609.02309#bib.bib9)\];R\-VLM \[[45](https://arxiv.org/html/2609.02309#bib.bib54)\];RegionFocus \[[42](https://arxiv.org/html/2609.02309#bib.bib11)\];ShowUI \[[35](https://arxiv.org/html/2609.02309#bib.bib42)\];DiMo\-GUI \[[55](https://arxiv.org/html/2609.02309#bib.bib55)\];ScreenSpot\-Pro \[[32](https://arxiv.org/html/2609.02309#bib.bib43)\];SimpAgent \[[11](https://arxiv.org/html/2609.02309#bib.bib44)\]\.Set\-of\-Mark \[[64](https://arxiv.org/html/2609.02309#bib.bib12)\];ScreenAI \[[6](https://arxiv.org/html/2609.02309#bib.bib13)\];OmniParser \[[40](https://arxiv.org/html/2609.02309#bib.bib14)\];Tree\-of\-Lens \[[19](https://arxiv.org/html/2609.02309#bib.bib10)\];GUI\-Actor \[[56](https://arxiv.org/html/2609.02309#bib.bib56)\];Agent\-S \[[4](https://arxiv.org/html/2609.02309#bib.bib15)\];Ferret\-UI 2 \[[34](https://arxiv.org/html/2609.02309#bib.bib16)\];UGround \[[20](https://arxiv.org/html/2609.02309#bib.bib17)\];Aria\-UI \[[66](https://arxiv.org/html/2609.02309#bib.bib19)\];Aguvis \[[62](https://arxiv.org/html/2609.02309#bib.bib18)\];UI\-TARS \[[46](https://arxiv.org/html/2609.02309#bib.bib20)\]\.Agent\-S \[[4](https://arxiv.org/html/2609.02309#bib.bib15)\];ColorBrowserAgent \[[75](https://arxiv.org/html/2609.02309#bib.bib45)\];GUI\-Rise \[[37](https://arxiv.org/html/2609.02309#bib.bib57)\];PAL\-UI \[[39](https://arxiv.org/html/2609.02309#bib.bib21)\];HiconAgent \[[77](https://arxiv.org/html/2609.02309#bib.bib22)\];SimpAgent \[[11](https://arxiv.org/html/2609.02309#bib.bib44)\];Read More, Think More \[[17](https://arxiv.org/html/2609.02309#bib.bib41)\]\.GUI\-KV \[[22](https://arxiv.org/html/2609.02309#bib.bib23)\];ST\-Lite \[[74](https://arxiv.org/html/2609.02309#bib.bib25)\];Continuous Memory \[[58](https://arxiv.org/html/2609.02309#bib.bib24)\];SecAgent \[[61](https://arxiv.org/html/2609.02309#bib.bib26)\]\.SkillWeaver \[[72](https://arxiv.org/html/2609.02309#bib.bib27)\];PolySkill \[[69](https://arxiv.org/html/2609.02309#bib.bib58)\];Mobile\-Agent\-E \[[54](https://arxiv.org/html/2609.02309#bib.bib28)\];ActionEngine \[[73](https://arxiv.org/html/2609.02309#bib.bib38)\];CoAct\-1 \[[50](https://arxiv.org/html/2609.02309#bib.bib53)\]\.Prune4Web \[[70](https://arxiv.org/html/2609.02309#bib.bib7)\];V\-Droid \[[14](https://arxiv.org/html/2609.02309#bib.bib29)\];VeriSafe Agent \[[30](https://arxiv.org/html/2609.02309#bib.bib47)\];GUI\-Shepherd \[[10](https://arxiv.org/html/2609.02309#bib.bib60)\];SenseAct \[[9](https://arxiv.org/html/2609.02309#bib.bib59)\];BacktrackAgent \[[57](https://arxiv.org/html/2609.02309#bib.bib30)\];LongHorizonUI \[[24](https://arxiv.org/html/2609.02309#bib.bib67)\]\.LASER \[[43](https://arxiv.org/html/2609.02309#bib.bib31)\];Auto\-Intent \[[27](https://arxiv.org/html/2609.02309#bib.bib32)\];OpenWebVoyager \[[21](https://arxiv.org/html/2609.02309#bib.bib33)\];GUI\-explorer \[[59](https://arxiv.org/html/2609.02309#bib.bib46)\];WebOperator \[[16](https://arxiv.org/html/2609.02309#bib.bib61)\];MobileUse \[[33](https://arxiv.org/html/2609.02309#bib.bib37)\]\.AndroidWorld \[[47](https://arxiv.org/html/2609.02309#bib.bib34)\];MMBench\-GUI \[[52](https://arxiv.org/html/2609.02309#bib.bib48)\];OSWorld\-Human \[[1](https://arxiv.org/html/2609.02309#bib.bib51)\];UI\-R1 \[[41](https://arxiv.org/html/2609.02309#bib.bib35)\];Think Twice, Click Once \[[51](https://arxiv.org/html/2609.02309#bib.bib36)\];GUI\-G1 \[[78](https://arxiv.org/html/2609.02309#bib.bib62)\];AdaGUI\-R1 \[[12](https://arxiv.org/html/2609.02309#bib.bib63)\];MobileUse \[[33](https://arxiv.org/html/2609.02309#bib.bib37)\];MobileWizard \[[36](https://arxiv.org/html/2609.02309#bib.bib49)\];AgentCPM\-GUI \[[71](https://arxiv.org/html/2609.02309#bib.bib50)\];Agent S2 \[[5](https://arxiv.org/html/2609.02309#bib.bib52)\];InfiGUIAgent \[[38](https://arxiv.org/html/2609.02309#bib.bib64)\]\.ActionEngine \[[73](https://arxiv.org/html/2609.02309#bib.bib38)\];CORE \[[18](https://arxiv.org/html/2609.02309#bib.bib39)\];GUIGuard \[[53](https://arxiv.org/html/2609.02309#bib.bib40)\];CoAct\-1 \[[50](https://arxiv.org/html/2609.02309#bib.bib53)\];Agent S2 \[[5](https://arxiv.org/html/2609.02309#bib.bib52)\];IntentCUA \[[31](https://arxiv.org/html/2609.02309#bib.bib65)\];OS\-Symphony \[[63](https://arxiv.org/html/2609.02309#bib.bib68)\];LongHorizonUI \[[24](https://arxiv.org/html/2609.02309#bib.bib67)\];UltraCUA \[[67](https://arxiv.org/html/2609.02309#bib.bib66)\]\.

Figure 2:Survey taxonomy of this review\-paper layout rooted at*Efficient GUI Agent*\. The left column gives the four major efficiency topics, the middle column preserves the existing subsection taxonomy, and the right column lists the papers integrated under each subtopic\.

## 3Observation Efficiency

Observation efficiency concerns how a GUI agent acquires a decision\-sufficient representation of the current interface while minimizing redundancy, grounding ambiguity, and downstream reasoning cost\. In GUI environments, inefficiency may arise from overly long textual interface representations, visually cluttered screenshots, or weakly structured multimodal observations\.

### 3\.1Textual Observation Reduction

For agents operating on the Document Object Model \(DOM\) or the accessibility tree \(AxTree\), the primary inefficiency arises from an overabundance of textual interface structure\. Real web pages often expose large interface trees containing decorative nodes, repeated text spans, layout artifacts, and task\-irrelevant branches, which substantially increase computational cost without improving action prediction[Abuelsaad et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib1)\. Agent\-E made this issue explicit in practical web\-agent design by advocating flexible DOM distillation and denoising as part of a hierarchical browser\-agent architecture, where raw DOM inputs can reach up to 800k tokens and typical tasks require 150–220 seconds and about 25 LLM calls[Abuelsaad et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib1)\. Beyond Pixels studies the issue directly through DOM downsampling, showing that aggressively compressed DOM snapshots can remain around the10310^\{3\}\-token order while preserving useful hierarchical signals for downstream decision making[Schiepanski and Piël \(2025\)](https://arxiv.org/html/2609.02309#bib.bib3)\. A more selective direction formulates reduction as retrieval rather than uniform truncation\. LineRetriever argues that the most useful lines are those that support future navigation decisions rather than those that are merely semantically similar to the goal text, reporting observation reductions of 61%, 72%, and 73% while retrieving up to 10 chunks of 100 tokens[Kerboua et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib5)\. FocusAgent selectively retrieves task\-relevant AxTree lines at each step, achieving more than 50% average AxTree reduction, often more than 80%, while capping context at 2k tokens[Kerboua et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib6)\. Prune4Web pushes this line further by moving pruning from inference\-time reading into explicit executable filtering programs, with examples where DOM trees of 10k–100k tokens and more than 500 candidate elements are reduced to fewer than 20 actionable candidates, thereby reducing both observation length and grounding search complexity[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib7)\.

### 3\.2Region\-Focused Visual Perception

When the primary observation is a screenshot, inefficiency is mainly an allocation problem: GUI images contain small actionable targets, repeated widgets, and irrelevant background regions\. SeeClick established the feasibility of screenshot\-only GUI agents while exposing visual grounding as a major bottleneck, although it does not report directly comparable token, latency, memory, or step savings[Cheng et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib8)\. Ferret\-UI and R\-VLM address this bottleneck by reallocating visual resolution through sub\-image partitioning, zoomed region proposals, and region\-aware grounding objectives; Ferret\-UI splits each screen into two sub\-images for any\-resolution processing, while R\-VLM reports about 5\.6 seconds per sample and up to2×2\\timesinference\-latency cost, showing that region proposals may shift cost into extra inference[You et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib9);[Park et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib54)\. RegionFocus and DiMo\-GUI make this focusing process adaptive at inference time, progressively zooming into task\-relevant or ambiguous regions instead of encoding the full screen uniformly; this improves visual allocation but can introduce new overhead, with RegionFocus reporting 66\.8% average trajectory overhead and a 19\.74% step\-count increase, and DiMo\-GUI using up to seven zoom iterations[Luo et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib11);[Wu et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib55)\. ShowUI applies the same principle at the token level, selecting UI\-relevant visual tokens to remove 33% redundant visual tokens and obtain a1\.4×1\.4\\timestraining speedup[Lin et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib42)\. ScreenSpot\-Pro and SimpAgent further show why such selective perception matters: professional high\-resolution GUI screens often exceed3​k×2​k3k\\times 2kresolution, while SimpAgent compresses history images into 64 tokens, uses 10–20\-token action outputs, and reports a 27% FLOP reduction in the LLM branch[Li et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib43);[Chen et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib44)\.

### 3\.3Observation Enrichment via Parsing and Hybridization

Observation efficiency is not only input reduction; it also depends on whether the observation is immediately actionable\. Screenshots preserve rendered context but lack structure, while DOM or AxTree representations are compact yet often incomplete, noisy, or misaligned with the rendered interface\. Set\-of\-Mark and ScreenAI address this gap by adding lightweight referential or textual annotations to screen elements; ScreenAI also illustrates the scale of screen\-understanding backbones, reporting 670M, 2B, and 5B model variants and input resolution up to8122812^\{2\}[Yang et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib12);[Baechler et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib13)\. OmniParser and Tree\-of\-Lens go further by converting screenshots or pointed regions into semi\-structured region, function, and layout representations that expose content and spatial relations for downstream reasoning; OmniParser adds two parser models and can restrict downstream reasoning to the top\-50 relevant elements[Lu et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib14);[Fan et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib10)\. GUI\-Actor complements this parser\-style enrichment with coordinate\-free action regions and a grounding verifier, adding about 20M parameters for a 2B model and 100M parameters for a 7B model so that action candidates can be generated in one forward pass while reducing brittle coordinate decoding[Wu et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib56)\. Hybrid systems such as Agent\-S and Ferret\-UI 2 combine rendered visual context with accessibility or cross\-platform structural signals, whereas UGround, Aria\-UI, Aguvis, and UI\-TARS show that strong screenshot\-first pipelines can also support grounding, planning, and action prediction without relying on full DOM or AxTree input: UGround uses about two\-thirds of the visual tokens required by a fixed1344×13441344\\times 1344setting, Aguvis reduces text\-agent inputs from roughly 4k–6k tokens per step to 1,196 tokens for a 70% input\-token reduction[Agashe et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib15);[Li et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib16);[Gou et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib17);[Yang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib19);[Xu et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib18);[Qin et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib20)\.

## 4Context and Memory Efficiency

Context and memory efficiency concerns how agents preserve, access, and update historical information over long interaction horizons without allowing context growth to dominate runtime and memory consumption\.

### 4\.1Summary\-Based Compression and Selective Look\-Back

As GUI trajectories grow longer, replaying past screenshots, actions, and reasoning traces becomes both costly and noisy\. Agent\-S, ColorBrowserAgent, and GUI\-Rise address this problem by replacing raw trajectory replay with compact task\-progress memories, progressive summaries, or progress\-aware summaries trained to support later action prediction; ColorBrowserAgent, for example, caps the interaction horizon at 30 steps, highlighting the need to prevent history growth from becoming unbounded[Agashe et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib15);[Zhou et al\. \(2026b\)](https://arxiv.org/html/2609.02309#bib.bib45);[Liu et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib57)\. More recent systems make this compression selective rather than purely lossy\. PAL\-UI combines dual\-level summarization with active look\-back, while HiconAgent and SimpAgent reduce redundant history through dynamic context sampling, anchor\-guided compression, or consistency\-guided pruning; HiconAgent reports 25\.21T FLOPs for compressed history compared with 35\.75T for an uncompressed 3B setting and 62\.31T for a 7B setting[Liu et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib21);[Zhou et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib22);[Chen et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib44)\. Read More, Think More[Enomoto et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib41)showing that diff\-based history can be more token\-efficient than replaying full prior observations, especially when WorkArena HTML pages can reach 40k–500k tokens under 4\-step or 9\-step look\-back settings\.

### 4\.2Runtime\-Level Representation Compression

A second family of methods addresses the memory bottleneck directly at the representation and serving level\. These methods optimize*how*the preserved information is represented and reused during inference\. GUI\-KV exploits the spatial and temporal redundancy of GUI trajectories, tailoring compression policies to GUI\-specific attention patterns; it reports 38\.9% fewer MFLOPs per decoded token at five screenshots, uses 5–20% cache budgets, and notes that five screenshots can exceed 80GB of GPU memory[Huang et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib23)\. ST\-Lite similarly targets the cache bottleneck with a training\-free KV\-compression strategy designed for long\-horizon GUI workloads, operating at 10–20% cache budgets and reporting a2\.45×2\.45\\timesdecoding speedup and1\.40×1\.40\\timesend\-to\-end speedup, although prefill remains close to1\.0×1\.0\\times, indicating that the savings are concentrated mainly on the decode side[Zhou et al\. \(2026a\)](https://arxiv.org/html/2609.02309#bib.bib25)\. These methods are especially important because they improve scalability even when the high\-level memory policy remains unchanged\.

Other work reduces context cost by replacing symbolic histories with denser internal representations\. Auto\-scaling Continuous Memory for GUI Agent replaces long textual summaries with fixed\-length continuous memory representations that preserve fine\-grained visual information, compressing each trajectory into eight embeddings even when raw trajectories exceed 15k tokens, with a reported data\-collection cost of about $4k and tuning of 1\.2% of parameters[Wu et al\. \(2025d\)](https://arxiv.org/html/2609.02309#bib.bib24)\. SecAgent takes a lighter\-weight route by distilling prior screenshots and actions into concise semantic context for efficient mobile control[Xie et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib26)\.

## 5Action Efficiency

Even when model inference is efficient, GUI agents can remain impractical because they waste interaction steps\. Action efficiency studies how to complete tasks with fewer actions, fewer irreversible mistakes, and less unproductive exploration\.

### 5\.1Action Abstraction

Action abstraction reduces step count by lifting repeated primitive operations into reusable skills, routines, or programs\. In the web setting, SkillWeaver discovers reusable website procedures and distills them into lightweight callable APIs[Zheng et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib27)\. PolySkill sharpens that idea from a transfer perspective by separating a skill’s abstract goal from its site\-specific implementation, with learned functions typically covering 2–5 GUI steps and allowing web agents to reuse skills across seen and unseen websites with fewer redundant exploratory actions[Yu et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib58)\. Mobile\-Agent\-E adopts a reusable*Shortcuts*with explicit preconditions, turning recurrent subtasks into more efficient and more robust routines[Wang et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib28)\. These methods share a stable intuition: if a control pattern recurs, the agent should not keep paying the full planning cost every time it appears\.

At a more programmatic extreme, ActionEngine replaces repeated reactive planning with state\-machine memory and executable programs, showing that frequent GUI interaction patterns can be compiled into reusable control structures; this compilation reduces cost from $0\.71 to $0\.06, latency from 237\.5 to 118\.3 seconds, input tokens from 62\.3k to 8\.1k, and model calls from 10\.2 to 1\.8[Zhong et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib38)\. CoAct\-1 expands abstraction beyond pure GUI actions by allowing coding to function as an execution modality, letting the system dynamically delegate a subtask either to a GUI operator or to a programmer agent that writes and executes code; this routing reduces average steps to 10\.15, compared with 15\.22 for GTA\-1 and 14\.90 for UI\-TARS[Song et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib53)\. This matters for action efficiency because some subtasks, especially OS\-level file manipulation or data processing, are inefficient when expressed as long GUI\-only trajectories\.

### 5\.2Action Pruning, Verification, and Recovery

A second route to action efficiency is to reduce or validate candidate actions before expensive reasoning or irreversible execution\. Prune4Web reduces both observation size and downstream action search by pruning DOM candidates early, with examples where more than 500 DOM elements shrink to fewer than 20 candidates[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib7)\. V\-Droid, ALTP and VeriSafe Agent score or verify candidate actions before execution to improve deployment efficiency and intent alignment: V\-Droid reports 2\.6k–8\.9k input tokens, 0\.7 seconds per decision, and 4\.3 seconds per step compared with typical mobile agents above 20 seconds per step, while VeriSafe Agent shows recovery examples requiring 1–3 actions and notes that tasks above 10 steps can exceed $1[Dai et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib29);[Lee et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib47);[Bai et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib2)\. GUI\-Shepherd learns process\-level rewards that can later be reused as a verifier signal during inference, although verifier\-call and latency metrics are not yet reported separately[Chen et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib60)\. SenseAct structures actions through typed commitments and post\-condition checks, reducing UI exposure by 65\.51% and reducing the need to repeatedly consult a large VLM for every low\-level execution decision[Cai et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib59)\.

Once actions are executed in long\-horizon settings, recovery becomes equally important\. BacktrackAgent explicitly introduces backtracking, together with verifier, judger, and reflector modules, to mitigate cascading failures after early mistakes[Wu et al\. \(2025c\)](https://arxiv.org/html/2609.02309#bib.bib30)\. LongHorizonUI combines reflective decision making with rollback\-oriented execution over average trajectories of 24\.6 steps and maximum trajectories of 37 steps[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67)\. The shared lesson is that action efficiency is not only about choosing fewer actions\. It is also about failing in ways that are detectable, reversible, and cheap to correct\.

### 5\.3Exploration Control

A third source of inefficiency is unstructured exploration in unfamiliar interfaces\. LASER models web interaction as state\-space exploration with explicit backtracking, making recovery from off\-trajectory decisions more systematic than in forward\-only prompting schemes[Ma et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib31)\. Auto\-Intent distills compact intents from demonstrations and uses them to guide self\-exploration, so that search is constrained by higher\-level latent goals rather than by unrestricted trial and error[Kim et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib32)\. OpenWebVoyager propose an exploration–feedback–optimization loop, treating exploration as a continual self\-improvement process rather than as a one\-off phase[He et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib33)\. GUI\-explorer further enriches this line by autonomously mining transition\-aware knowledge and function\-aware trajectories[Xie et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib46)\. WebOperator combines pre\-execution filtering, best\-first search, and safe backtracking so that the agent does not pay the full price of blind rollout in complex web tasks[Dihan et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib61)\.MobileUse propose a system where proactive exploration is invoked to handle cold\-start mobile environments rather than as an always\-on behavior[Li et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib37)\. Across these works, the shared lesson is that exploration becomes action\-efficient only when it is guided by reusable structure, predicted intent, or learned transition logic\.

## 6Planner\-Side and System Efficiency

Planner\-side efficiency concerns the computational burden inside the decision loop, while system efficiency concerns the broader runtime around the planner, including memory reuse, privacy\-aware routing, and orchestration across heterogeneous components\.

### 6\.1Planner\-Side Efficiency

Recent benchmark work makes clear that efficiency is not reducible to final task success\. MMBench\-GUI introduce an explicitly efficiency\-aware evaluation perspective across multiple platforms, using a 50\-step budget and reporting redundant\-step cost values of 7–8, privacy noise of 40%, and specialist cost of 16%[Wang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib48)\. OSWorld\-Human sharpens the diagnosis further by showing that planning and reflection dominate end\-to\-end latency in computer\-use agents, with agents requiring2\.7×2\.7\\times–4\.3×4\.3\\timesmore steps than human\-derived trajectories and reflection accounting for 76%–96% of task latency[Abhyankar et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib51)\. These benchmark\-driven analyses are important because they motivate planner\-side efficiency as a first\-class systems objective rather than an afterthought\.

Methodologically, UI\-R1 shows that rule\-based reinforcement learning can improve GUI action prediction efficiently even in relatively compact models, using only 136 training samples and about 8 hours on 8 RTX 4090 GPUs, while also exposing OOM risk under high\-pixel settings[Lu et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib35)\. Think Twice, Click Once argues that grounding and decision making should not use uniform reasoning depth; instead, fast and slow modes should be deployed selectively according to task difficulty, even though invoking slow thinking raises processing time from 2\.6 to 5\.4 seconds and its data include 300k examples with 150k slow samples[Tang et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib36)\. GUI\-G1 analyzes R1\-style GUI grounding pipelines and finds that longer chains of thought can hurt rather than help\. AdaGUI\-R1 reports a similar lesson from adaptive reasoning scheduling, reducing unnecessary reasoning tokens by 40% while adding 23\.5% FLOPs under its harder\-example schedule, suggesting that selective thinking is likely to remain central in mobile and cross\-app control[Chen et al\. \(2025c\)](https://arxiv.org/html/2609.02309#bib.bib63)\.

Other methods factorize planning rather than merely shortening it\. MobileUse uses hierarchical reflection invoked on demand rather than at every step, reducing reflection overhead to 10% after exploration that can cost up to 100 steps per app at 19\.5 seconds per step[Li et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib37)\. MobileWizard combines structured reasoning with progressive reinforcement learning to achieve strong performance from modest data budgets, using 24\.5k public trajectories plus 300 remedial trajectories and fewer than 50k trajectories overall[Lin et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib49)\. AgentCPM\-GUI emphasizes efficient deployment through grounding\-aware training and a compact action space, trained on 470k atomic steps with about 8\.5 steps per trajectory, while retaining the last four actions/images and using a max\_new\_tokens setting of 2048[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib50)\. Agent S2 reframes planner efficiency compositionally, delegating different cognitive roles across generalist and specialist modules to avoid overloading a single monolithic planner with every subproblem[Agashe et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib52)\. InfiGUIAgent adds another perspective by showing that a compact generalist model can integrate native reasoning and reflection without fully externalizing every planner role into a separate subsystem[Liu et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib64)\.

### 6\.2System Efficiency Beyond the Planner

System efficiency depends on the runtime around the planner, not only on the planner itself\. ActionEngine amortizes repeated online reasoning through state\-machine memory and program synthesis, while CoAct\-1 routes subtasks between GUI interaction and code execution when different backends offer lower cost[Zhong et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib38);[Song et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib53)\. CORE and GUIGuard make deployment topology part of the efficiency problem by splitting work across local/cloud models and adding explicit privacy\-recognition or privacy\-protection stages[Fan et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib39);[Wang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib40)\. IntentCUA reduces redundant replanning through intent\-level abstractions and shared plan memory across collaborative desktop workflows[Lee et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib65)\. OS\-Symphony and LongHorizonUI extend this systems view to long\-horizon robustness, combining reflection, memory, tutorial retrieval, element\-indexed perception, or rollback\-based execution to keep trajectories recoverable in unseen or sustained tasks[Yang et al\. \(2026a\)](https://arxiv.org/html/2609.02309#bib.bib68);[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67)\. The central lesson is that efficient GUI agency requires joint accounting of observation design, memory placement, execution routing, verification, and orchestration cost across the full runtime\.

## 7Open Challenges and Future Directions

Several open problems follow from this systems view\. First, the field still lacks honest efficiency accounting: many papers report local savings in tokens, actions, or module latency, but benchmarks often stop at success rate or aggregate completion, making cross\-paper comparison difficult\. OSWorld\-Human and MMBench\-GUI move in the right direction by profiling efficiency more directly, yet verifier calls, parser overhead, and multi\-agent orchestration still lack a shared pricing framework[Abhyankar et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib51);[Wang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib48)\. The second challenge is that there are fewer GUI\-agent benchmarks\. Future benchmarks should report peak GPU memory, prefill and decode latency, MFLOPs per decoded token, GPU\-hours for training or search, and success\-normalized GPU cost, so that the community can evaluate whether an agent is deployable under realistic serving constraints rather than merely efficient in tokens or steps\.

## References

- Abhyankaret al\.\(2025\)R\. Abhyankar, Q\. Qi, and Y\. ZhangOSWorld\-human: benchmarking the efficiency of computer\-use agents\.arXiv preprint arXiv:2506\.16042\.External Links:[Link](https://arxiv.org/abs/2506.16042)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.5.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p2.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.3.1),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p1.1),[§7](https://arxiv.org/html/2609.02309#S7.p1.1)\.
- Abhyankaret al\.\(2026\)R\. Abhyankar, Q\. Qi, and Y\. ZhangOSWorld\-human: benchmarking the efficiency of computer\-use agents\.External Links:2506\.16042,[Link](https://arxiv.org/abs/2506.16042)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p2.1)\.
- Abuelsaadet al\.\(2024\)T\. Abuelsaad, D\. Akkil, P\. Dey, A\. Jagmohan, A\. Vempaty, and R\. KokkuAgent\-e: from autonomous web navigation to foundational design principles in agentic systems\.arXiv preprint arXiv:2407\.13032\.External Links:[Link](https://arxiv.org/abs/2407.13032)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.3.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.16.2.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§3\.1](https://arxiv.org/html/2609.02309#S3.SS1.p1.1)\.
- Agasheet al\.\(2024\)S\. Agashe, J\. Han, S\. Gan, J\. Yang, A\. Li, and X\. E\. WangAgent s: an open agentic framework that uses computers like a human\.arXiv preprint arXiv:2410\.08164\.External Links:[Link](https://arxiv.org/abs/2410.08164)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.24.1.1.1),[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.3.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.3.3),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.2.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Agasheet al\.\(2025\)S\. Agashe, K\. Wong, V\. Tu, J\. Yang, A\. Li, and X\. E\. WangAgent s2: a compositional generalist\-specialist framework for computer use agents\.arXiv preprint arXiv:2504\.00906\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2504.00906),[Link](https://arxiv.org/abs/2504.00906)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.13.1.1.1),[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.20.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.6.2),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.3.2),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p3.1)\.
- Baechleret al\.\(2024\)G\. Baechler, S\. Sunkara, M\. Wang, F\. Zubach, H\. Mansoor, V\. Etter, V\. Cărbune, J\. Lin, J\. Chen, and A\. SharmaScreenAI: a vision\-language model for ui and infographics understanding\.arXiv preprint arXiv:2402\.04615\.External Links:[Link](https://arxiv.org/abs/2402.04615)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.20.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.2.2),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Baiet al\.\(2025\)B\. Bai, J\. Cao, Y\. Luo, and T\. ChenLocal information matters: inference acceleration for grounded conversation generation models through adaptive local\-aware token pruning\.External Links:2503\.23959,[Link](https://arxiv.org/abs/2503.23959)Cited by:[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p1.1)\.
- Bonattiet al\.\(2024\)R\. Bonatti, D\. Zhao, F\. Bonacci, D\. Dupont, S\. Abdali, Y\. Li, Y\. Lu, J\. Wagle, K\. Koishida, A\. Bucker, L\. Jang, and Z\. HuiWindows Agent Arena: evaluating multi\-modal os agents at scale\.arXiv preprint arXiv:2409\.08264\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2409.08264),[Link](https://arxiv.org/abs/2409.08264)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1)\.
- Caiet al\.\(2026\)H\. Cai, T\. Ma, J\. Shao, T\. Tang, I\. Tsang, Y\. Lyu, and H\. YinSenseAct: structuring gui actions for reliable planning and verification\.Note:OpenReview, ICLR 2026 AIWILD workshopExternal Links:[Link](https://openreview.net/forum?id=0DznNQFW4g)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.13.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.3.2),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p1.1)\.
- Chenet al\.\(2025a\)C\. Chen, K\. Ji, H\. Zhong, M\. Zhu, A\. Li, G\. Gan, Z\. Huang, C\. Zou, J\. Liu, J\. Chen, H\. Chen, and C\. ShenGUI\-shepherd: reliable process reward and verification for long\-sequence gui tasks\.Note:OpenReview, submitted to ICLR 2026External Links:[Link](https://openreview.net/forum?id=9hM4YRMhfT)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.12.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.3.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p1.1)\.
- Chenet al\.\(2025b\)G\. Chen, X\. Zhou, R\. Shao, Y\. Lyu, K\. Zhou, S\. Wang, W\. Li, Y\. Li, Z\. Qi, and L\. NieLess is more: empowering gui agent with context\-aware simplification\.InProceedings of the IEEE/CVF International Conference on Computer Vision,pp\. 5901–5911\.External Links:[Link](https://openaccess.thecvf.com/content/ICCV2025/html/Chen_Less_is_More_Empowering_GUI_Agent_with_Context-Aware_Simplification_ICCV_2025_paper.html)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.17.1.1.1),[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.8.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.4.2),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.4.1),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Chenet al\.\(2025c\)J\. Chen, R\. Lv, H\. Jing, Z\. Dang, S\. Fang, C\. Ma, L\. Zhao, and J\. TengDifficulty\-aware reasoning for mobile gui automation via reinforcement fine\-tuning\.Note:OpenReview, submitted to ICLR 2026External Links:[Link](https://openreview.net/forum?id=Ric2If6Xur)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.9.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.5.1),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p2.1)\.
- Chenget al\.\(2024\)K\. Cheng, Q\. Sun, Y\. Chu, F\. Xu, Y\. Li, J\. Zhang, and Z\. WuSeeClick: harnessing gui grounding for advanced visual gui agents\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 9313–9332\.External Links:[Link](https://aclanthology.org/2024.acl-long.505/)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.10.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.2.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Daiet al\.\(2025\)G\. Dai, S\. Jiang, T\. Cao, Y\. Li, Y\. Yang, R\. Tan, M\. Li, and L\. QiuAdvancing mobile gui agents: a verifier\-driven approach to practical deployment\.arXiv preprint arXiv:2503\.15937\.External Links:[Link](https://arxiv.org/abs/2503.15937)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.10.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.2.2),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p1.1)\.
- Denget al\.\(2023\)X\. Deng, Y\. Gu, B\. Zheng, S\. Chen, S\. Stevens, B\. Wang, H\. Sun, and Y\. SuMind2Web: towards a generalist agent for the web\.arXiv preprint arXiv:2306\.06070\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2306.06070),[Link](https://arxiv.org/abs/2306.06070)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[§2\.1](https://arxiv.org/html/2609.02309#S2.SS1.p1.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1)\.
- Dihanet al\.\(2025\)M\. L\. Dihan, T\. Hashem, M\. E\. Ali, and M\. R\. ParvezWebOperator: action\-aware tree search for autonomous agents in web environment\.Note:OpenReview, submitted to ICLR 2026External Links:[Link](https://openreview.net/forum?id=vnEuxLVFmN)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.21.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.23.3.2),[§5\.3](https://arxiv.org/html/2609.02309#S5.SS3.p1.1)\.
- Enomotoet al\.\(2026\)M\. Enomoto, R\. Obara, H\. Zhang, and M\. OyamadaRead more, think more: revisiting observation reduction for web agents\.arXiv preprint arXiv:2604\.01535\.External Links:[Link](https://arxiv.org/abs/2604.01535)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.8.1.1.1),[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.9.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.16.4.2),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.4.2),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Fanet al\.\(2025\)G\. Fan, C\. Niu, C\. Lyu, F\. Wu, and G\. ChenCORE: reducing ui exposure in mobile agents via collaboration between cloud and local llms\.InAdvances in Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=klOr9y9nMU)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.17.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.2.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Fanet al\.\(2024\)Y\. Fan, L\. Ding, C\. Kuo, S\. Jiang, Y\. Zhao, X\. Guan, J\. Yang, Y\. Zhang, and X\. E\. WangRead anywhere pointed: layout\-aware gui screen reading with tree\-of\-lens grounding\.arXiv preprint arXiv:2406\.19263\.External Links:[Link](https://arxiv.org/abs/2406.19263)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.22.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.3.1),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Gouet al\.\(2024\)B\. Gou, R\. Wang, B\. Zheng, Y\. Xie, C\. Chang, Y\. Shu, H\. Sun, and Y\. SuNavigating the digital world as humans do: universal visual grounding for gui agents\.arXiv preprint arXiv:2410\.05243\.External Links:[Link](https://arxiv.org/abs/2410.05243)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.26.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.4.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Heet al\.\(2024\)H\. He, W\. Yao, K\. Ma, W\. Yu, H\. Zhang, T\. Fang, Z\. Lan, and D\. YuOpenWebVoyager: building multimodal web agents via iterative real\-world exploration, feedback and optimization\.arXiv preprint arXiv:2410\.19609\.External Links:[Link](https://arxiv.org/abs/2410.19609)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.19.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.23.2.3),[§5\.3](https://arxiv.org/html/2609.02309#S5.SS3.p1.1)\.
- Huanget al\.\(2025\)K\. Huang, H\. Qiu, Y\. Dai, C\. Xiong, and C\. WuGUI\-kv: efficient gui agents via kv cache with spatio\-temporal awareness\.arXiv preprint arXiv:2510\.00536\.External Links:[Link](https://arxiv.org/abs/2510.00536)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.11.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p2.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.20.2.1),[§4\.2](https://arxiv.org/html/2609.02309#S4.SS2.p1.1)\.
- Jinet al\.\(2024\)Y\. Jin, J\. Li, Y\. Liu, T\. Gu, K\. Wu, Z\. Jiang, M\. He, B\. Zhao, X\. Tan, Z\. Gan, Y\. Wang, C\. Wang, and L\. MaEfficient multimodal large language models: a survey\.arXiv preprint arXiv:2405\.10739\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2405.10739),[Link](https://arxiv.org/abs/2405.10739)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p5.1)\.
- Kanget al\.\(2026\)B\. Kang, S\. Wen, Y\. Bi, S\. Wu, X\. Yuan, R\. Shao, J\. Wang, and Z\. TianLongHorizonUI: a unified framework for robust long\-horizon task automation of gui agent\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=BK7Mk5d4WE)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.15.1.1.1),[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.23.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.4.2),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.4.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p2.1),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Kerbouaet al\.\(2025a\)I\. Kerboua, S\. Omidi Shayegan, X\. H\. Lù, L\. Boisvert, M\. Thakkar, M\. Caccia, J\. Espinas, A\. Aussem, V\. Eglin, and A\. LacosteFocusAgent: simple yet effective ways of trimming the large context of web agents\.arXiv preprint arXiv:2510\.03204\.External Links:[Link](https://arxiv.org/abs/2510.03204)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.6.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.16.3.2),[§3\.1](https://arxiv.org/html/2609.02309#S3.SS1.p1.1)\.
- Kerbouaet al\.\(2025b\)I\. Kerboua, S\. Omidi Shayegan, M\. Thakkar, X\. H\. Lù, M\. Caccia, V\. Eglin, A\. Aussem, J\. Espinas, and A\. LacosteLineRetriever: planning\-aware observation reduction for web agents\.arXiv preprint arXiv:2507\.00210\.External Links:[Link](https://arxiv.org/abs/2507.00210)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.5.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.16.3.1),[§3\.1](https://arxiv.org/html/2609.02309#S3.SS1.p1.1)\.
- Kimet al\.\(2024\)J\. Kim, D\. Kim, L\. Logeswaran, S\. Sohn, and H\. LeeAuto\-intent: automated intent discovery and self\-exploration for large language model web agents\.arXiv preprint arXiv:2410\.22552\.External Links:[Link](https://arxiv.org/abs/2410.22552)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.18.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.23.2.2),[§5\.3](https://arxiv.org/html/2609.02309#S5.SS3.p1.1)\.
- Kohet al\.\(2024\)J\. Y\. Koh, R\. Lo, L\. Jang, V\. Duvvur, M\. C\. Lim, P\. Huang, G\. Neubig, S\. Zhou, R\. Salakhutdinov, and D\. FriedVisualWebArena: evaluating multimodal agents on realistic visual web tasks\.arXiv preprint arXiv:2401\.13649\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2401.13649),[Link](https://arxiv.org/abs/2401.13649)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1)\.
- Le Sellier De Chezelleset al\.\(2024\)T\. Le Sellier De Chezelles, M\. Gasse, A\. Drouin, M\. Caccia, L\. Boisvert, M\. Thakkar, T\. Marty, R\. Assouel, S\. O\. Shayegan, L\. K\. Jang, X\. H\. Lù, O\. Yoran, D\. Kong, F\. F\. Xu, S\. Reddy, Q\. Cappart, G\. Neubig, R\. Salakhutdinov, N\. Chapados, and A\. LacosteThe BrowserGym ecosystem for web agent research\.arXiv preprint arXiv:2412\.05467\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2412.05467),[Link](https://arxiv.org/abs/2412.05467)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1)\.
- Leeet al\.\(2025\)J\. Lee, D\. Lee, C\. Choi, Y\. Im, J\. Wi, K\. Heo, S\. Oh, S\. Lee, and I\. ShinVeriSafe agent: safeguarding mobile gui agent via logic\-based action verification\.arXiv preprint arXiv:2503\.18492\.External Links:[Link](https://arxiv.org/abs/2503.18492)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.11.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.2.3),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p1.1)\.
- Leeet al\.\(2026\)S\. Lee, S\. Yoon, S\. Lee, D\. Park, D\. Kim, Y\. Chun, and J\. Y\. SimIntentCUA: learning intent\-level representations for skill abstraction and multi\-agent planning in computer\-use agents\.Note:OpenReview, AAMAS 2026 full paperExternal Links:[Link](https://openreview.net/forum?id=qeSsHx2wIr)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.21.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.3.3),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Liet al\.\(2025a\)K\. Li, Z\. Meng, H\. Lin, Z\. Luo, Y\. Tian, J\. Ma, Z\. Huang, and T\. ChuaScreenSpot\-pro: gui grounding for professional high\-resolution computer use\.arXiv preprint arXiv:2504\.07981\.External Links:[Link](https://arxiv.org/abs/2504.07981)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.16.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.4.1),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Liet al\.\(2025b\)N\. Li, X\. Qu, J\. Zhou, J\. Wang, M\. Wen, K\. Du, X\. Lou, Q\. Peng, J\. Wang, and W\. ZhangMobileUse: a hierarchical reflection\-driven gui agent for autonomous mobile operation\.Note:OpenReview, NeurIPS 2025 posterExternal Links:[Link](https://openreview.net/forum?id=KR6tnkb6h4)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.22.1.1.1),[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.10.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.23.4.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.5.2),[§5\.3](https://arxiv.org/html/2609.02309#S5.SS3.p1.1),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p3.1)\.
- Liet al\.\(2024\)Z\. Li, K\. You, H\. Zhang, D\. Feng, H\. Agrawal, X\. Li, M\. P\. S\. Moorthy, J\. Nichols, Y\. Yang, and Z\. GanFerret\-ui 2: mastering universal user interface understanding across platforms\.arXiv preprint arXiv:2410\.18967\.External Links:[Link](https://arxiv.org/abs/2410.18967)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.25.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.4.1),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Linet al\.\(2025a\)K\. Q\. Lin, L\. Li, D\. Gao, Z\. Yang, S\. Wu, Z\. Bai, W\. Lei, L\. Wang, and M\. Z\. ShouShowUI: one vision\-language\-action model for gui visual agent\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 19498–19508\.External Links:[Document](https://dx.doi.org/10.1109/CVPR52734.2025.01816),[Link](https://openaccess.thecvf.com/content/CVPR2025/html/Lin_ShowUI_One_Vision-Language-Action_Model_for_GUI_Visual_Agent_CVPR_2025_paper.html)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.14.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.3.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Linet al\.\(2025b\)W\. Lin, Y\. Chai, H\. Xiao, L\. Bian, G\. Liu, L\. Liu, S\. Ren, P\. Shi, Y\. Wen, X\. Chen, A\. Zhou, and H\. LiMobileWizard: a data\-efficient gui agent with structured reasoning and progressive reinforcement learning\.Note:OpenReviewExternal Links:[Link](https://openreview.net/forum?id=Aobvdp3XmP)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.11.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.5.3),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p3.1)\.
- Liuet al\.\(2025a\)T\. Liu, C\. Wang, R\. Li, Y\. Yu, X\. He, and S\. BaiGUI\-rise: structured reasoning and history summarization for gui navigation\.Note:OpenReview, NeurIPS 2025 posterExternal Links:[Link](https://openreview.net/forum?id=YMPYLesItf)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.5.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.3.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Liuet al\.\(2026\)Y\. Liu, P\. Li, Z\. Wei, C\. Xie, X\. Hu, X\. Xu, S\. Zhang, X\. Han, H\. Yang, and F\. WuInfiGUIAgent: a multimodal generalist gui agent with native reasoning and reflection\.InProceedings of the 19th Conference of the European Chapter of the Association for Computational Linguistics \(Volume 1: Long Papers\),pp\. 1035–1051\.External Links:[Document](https://dx.doi.org/10.18653/v1/2026.eacl-long.45),[Link](https://aclanthology.org/2026.eacl-long.45/)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.14.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.7.1),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p3.1)\.
- Liuet al\.\(2025b\)Z\. Liu, J\. Li, W\. X\. Zhao, D\. Gao, Y\. Li, and J\. WenPAL\-ui: planning with active look\-back for vision\-based gui agents\.arXiv preprint arXiv:2510\.00413\.External Links:[Link](https://arxiv.org/abs/2510.00413)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.6.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.3.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Luet al\.\(2024\)Y\. Lu, J\. Yang, Y\. Shen, and A\. AwadallahOmniParser for pure vision based gui agent\.arXiv preprint arXiv:2408\.00203\.External Links:[Link](https://arxiv.org/abs/2408.00203)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.21.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.2.3),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Luet al\.\(2025\)Z\. Lu, Y\. Chai, Y\. Guo, X\. Yin, L\. Liu, H\. Wang, H\. Xiao, S\. Ren, G\. Xiong, and H\. LiUI\-r1: enhancing efficient action prediction of gui agents by reinforcement learning\.arXiv preprint arXiv:2503\.21620\.External Links:[Link](https://arxiv.org/abs/2503.21620)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.6.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.3.2),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p2.1)\.
- Luoet al\.\(2025\)T\. Luo, L\. Logeswaran, J\. Johnson, and H\. LeeVisual test\-time scaling for gui agent grounding\.arXiv preprint arXiv:2505\.00684\.External Links:[Link](https://arxiv.org/abs/2505.00684)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.13.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.3.1),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Maet al\.\(2023\)K\. Ma, H\. Zhang, H\. Wang, X\. Pan, and D\. YuLASER: llm agent with state\-space exploration for web navigation\.arXiv preprint arXiv:2309\.08172\.External Links:[Link](https://arxiv.org/abs/2309.08172)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.17.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.23.2.1),[§5\.3](https://arxiv.org/html/2609.02309#S5.SS3.p1.1)\.
- Nguyenet al\.\(2024\)D\. Nguyen, J\. Chen, Y\. Wang, G\. Wu, N\. Park, Z\. Hu, H\. Lyu, J\. Wu, R\. Aponte, Y\. Xia, X\. Li, J\. Shi, H\. Chen, V\. D\. Lai, Z\. Xie, S\. Kim, R\. Zhang, T\. Yu, M\. Tanjim, N\. K\. Ahmed, P\. Mathur, S\. Yoon, L\. Yao, B\. Kveton, J\. Kil, T\. H\. Nguyen, T\. Bui, T\. Zhou, R\. A\. Rossi, and F\. DernoncourtGUI agents: a survey\.arXiv preprint arXiv:2412\.13501\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2412.13501),[Link](https://arxiv.org/abs/2412.13501)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p3.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1)\.
- Parket al\.\(2025\)J\. Park, P\. Tang, S\. Das, S\. Appalaraju, K\. Y\. Singh, R\. Manmatha, and S\. GhadarR\-vlm: region\-aware vision language model for precise gui grounding\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 9669–9685\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.501),[Link](https://aclanthology.org/2025.findings-acl.501/)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.12.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.2.3),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Qinet al\.\(2025\)Y\. Qin, Y\. Ye, J\. Fang, H\. Wang, S\. Liang, S\. Tian, J\. Zhang, J\. Li, Y\. Li, S\. Huang, W\. Zhong, K\. Li, J\. Yang, Y\. Miao, W\. Lin, L\. Liu, X\. Jiang, Q\. Ma, J\. Li, X\. Xiao, K\. Cai, C\. Li, Y\. Zheng, C\. Jin, C\. Li, X\. Zhou, M\. Wang, H\. Chen, Z\. Li, H\. Yang, H\. Liu, F\. Lin, T\. Peng, X\. Liu, and G\. ShiUI\-tars: pioneering automated gui interaction with native agents\.arXiv preprint arXiv:2501\.12326\.External Links:[Link](https://arxiv.org/abs/2501.12326)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.29.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.5.2),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Rawleset al\.\(2024\)C\. Rawles, S\. Clinckemaillie, Y\. Chang, J\. Waltz, G\. Lau, M\. Fair, A\. Li, W\. Bishop, W\. Li, F\. Campbell\-Ajala, D\. Toyama, R\. Berry, D\. Tyamagundlu, T\. Lillicrap, and O\. RivaAndroidWorld: a dynamic benchmarking environment for autonomous agents\.arXiv preprint arXiv:2405\.14573\.External Links:[Link](https://arxiv.org/abs/2405.14573)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.3.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.2.1),[§2\.1](https://arxiv.org/html/2609.02309#S2.SS1.p1.2)\.
- Sageret al\.\(2025\)P\. J\. Sager, B\. Meyer, P\. Yan, R\. von Wartburg\-Kottler, L\. Etaiwi, A\. Enayati, G\. Nobel, A\. Abdulkadir, B\. F\. Grewe, and T\. StadelmannA comprehensive survey of agents for computer use: foundations, challenges, and future directions\.arXiv preprint arXiv:2501\.16150\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2501.16150),[Link](https://arxiv.org/abs/2501.16150)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p3.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[§2\.1](https://arxiv.org/html/2609.02309#S2.SS1.p1.1)\.
- Schiepanski and Piël \(2025\)T\. M\. Schiepanski and N\. PiëlBeyond pixels: exploring dom downsampling for llm\-based web agents\.arXiv preprint arXiv:2508\.04412\.External Links:[Link](https://arxiv.org/abs/2508.04412)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.4.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.16.2.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§3\.1](https://arxiv.org/html/2609.02309#S3.SS1.p1.1)\.
- Songet al\.\(2026\)L\. Song, Y\. Dai, V\. Prabhu, J\. Zhang, T\. Shi, L\. Li, J\. Li, S\. Savarese, Z\. Chen, J\. Zhao, R\. Xu, and C\. XiongCoAct\-1: computer\-using multi\-agent system with coding actions\.Note:OpenReview, ICLR 2026 posterExternal Links:[Link](https://openreview.net/forum?id=l1MQVgIKEU)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.7.1.1.1),[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.19.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.21.4.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.3.1),[§5\.1](https://arxiv.org/html/2609.02309#S5.SS1.p2.1),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Tanget al\.\(2025\)F\. Tang, Y\. Shen, H\. Zhang, S\. Chen, G\. Hou, W\. Zhang, W\. Zhang, K\. Song, W\. Lu, and Y\. ZhuangThink twice, click once: enhancing gui grounding via fast and slow systems\.arXiv preprint arXiv:2503\.06470\.External Links:[Link](https://arxiv.org/abs/2503.06470)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.7.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.4.1),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p2.1)\.
- Wanget al\.\(2025a\)X\. Wang, Z\. Wu, J\. Xie, Z\. Ding, B\. Yang, Z\. Li, Z\. Liu, Q\. Li, X\. Dong, Z\. Chen, W\. Wang, X\. Zhao, J\. Chen, H\. Duan, T\. Xie, C\. Yang, S\. Su, Y\. Yu, Y\. Huang, Y\. Liu, X\. Zhang, Y\. Zhang, X\. Yue, W\. Su, X\. Zhu, W\. Shen, J\. Dai, and W\. WangMMBench\-gui: hierarchical multi\-platform evaluation framework for gui agents\.arXiv preprint arXiv:2507\.19478\.External Links:[Link](https://arxiv.org/abs/2507.19478)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.4.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.2.2),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p1.1),[§7](https://arxiv.org/html/2609.02309#S7.p1.1)\.
- Wanget al\.\(2026\)Y\. Wang, Z\. Zhang, W\. Zhou, W\. Zhang, J\. Zhang, Q\. Zhu, Y\. Shi, S\. Zheng, and J\. HeGUIGuard: toward a general framework for privacy\-preserving gui agents\.arXiv preprint arXiv:2601\.18842\.External Links:[Link](https://arxiv.org/abs/2601.18842)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.18.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.2.3),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Wanget al\.\(2025b\)Z\. Wang, H\. Xu, J\. Wang, X\. Zhang, M\. Yan, J\. Zhang, F\. Huang, and H\. JiMobile\-agent\-e: self\-evolving mobile assistant for complex tasks\.arXiv preprint arXiv:2501\.11733\.External Links:[Link](https://arxiv.org/abs/2501.11733)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.5.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.21.3.1),[§5\.1](https://arxiv.org/html/2609.02309#S5.SS1.p1.1)\.
- Wuet al\.\(2025a\)H\. Wu, H\. Chen, Y\. Cai, C\. Liu, Q\. Ye, M\. Yang, and Y\. WangDiMo\-gui: advancing test\-time scaling in gui grounding via modality\-aware visual reasoning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 26246–26256\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1334),[Link](https://aclanthology.org/2025.emnlp-main.1334/)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.15.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.3.3),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Wuet al\.\(2025b\)Q\. Wu, K\. Cheng, R\. Yang, C\. Zhang, J\. Yang, H\. Jiang, J\. Mu, B\. Peng, B\. Qiao, R\. Tan, S\. Qin, L\. Liden, Q\. Lin, H\. Zhang, T\. Zhang, J\. Zhang, D\. Zhang, and J\. GaoGUI\-actor: coordinate\-free visual grounding for gui agents\.Note:NeurIPS 2025 posterExternal Links:[Document](https://dx.doi.org/10.48550/arXiv.2506.03143),[Link](https://arxiv.org/abs/2506.03143)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.23.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.3.2),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Wuet al\.\(2025c\)Q\. Wu, P\. Gao, W\. Liu, and J\. LuanBacktrackAgent: enhancing gui agent with error detection and backtracking mechanism\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,pp\. 4250–4272\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.212),[Link](https://aclanthology.org/2025.emnlp-main.212/)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.14.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.4.1),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p2.1)\.
- Wuet al\.\(2025d\)W\. Wu, K\. Zhou, R\. Yuan, V\. Yu, S\. Wang, Z\. Hu, and B\. HuangAuto\-scaling continuous memory for gui agent\.arXiv preprint arXiv:2510\.09038\.External Links:[Link](https://arxiv.org/abs/2510.09038)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.13.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.20.3.1),[§4\.2](https://arxiv.org/html/2609.02309#S4.SS2.p2.1)\.
- Xieet al\.\(2025\)B\. Xie, R\. Shao, G\. Chen, K\. Zhou, Y\. Li, J\. Liu, M\. Zhang, and L\. NieGUI\-explorer: autonomous exploration and mining of transition\-aware knowledge for gui agent\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics,External Links:[Link](https://aclanthology.org/2025.acl-long.282/)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.20.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.23.3.1),[§5\.3](https://arxiv.org/html/2609.02309#S5.SS3.p1.1)\.
- Xieet al\.\(2024\)T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei, Y\. Liu, Y\. Xu, S\. Zhou, S\. Savarese, C\. Xiong, V\. Zhong, and T\. YuOSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.arXiv preprint arXiv:2404\.07972\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2404.07972),[Link](https://arxiv.org/abs/2404.07972)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[§2\.1](https://arxiv.org/html/2609.02309#S2.SS1.p1.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1)\.
- Xieet al\.\(2026\)Y\. Xie, S\. Chen, J\. Xing, W\. Jiang, Z\. Zhu, Y\. Wang, P\. Bu, J\. Song, Y\. Jiang, and B\. ZhengSecAgent: efficient mobile gui agent with semantic context\.arXiv preprint arXiv:2603\.08533\.External Links:[Link](https://arxiv.org/abs/2603.08533)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.14.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.20.3.2),[§4\.2](https://arxiv.org/html/2609.02309#S4.SS2.p2.1)\.
- Xuet al\.\(2024\)Y\. Xu, Z\. Wang, J\. Wang, D\. Lu, T\. Xie, A\. Saha, D\. Sahoo, T\. Yu, and C\. XiongAguvis: unified pure vision agents for autonomous gui interaction\.arXiv preprint arXiv:2412\.04454\.External Links:[Link](https://arxiv.org/abs/2412.04454)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.28.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.5.1),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Yanget al\.\(2026a\)B\. Yang, K\. Jin, Z\. Wu, Z\. Liu, Q\. Sun, Z\. Li, J\. Xie, Z\. Liu, F\. Xu, K\. Cheng, Q\. Li, Y\. Wang, Y\. Qiao, Z\. Wang, and Z\. DingOS\-symphony: a holistic framework for robust and generalist computer\-using agent\.arXiv preprint arXiv:2601\.07779\.External Links:[Link](https://arxiv.org/abs/2601.07779)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.22.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.4.1),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Yanget al\.\(2023\)J\. Yang, H\. Zhang, F\. Li, X\. Zou, C\. Li, and J\. GaoSet\-of\-mark prompting unleashes extraordinary visual grounding in gpt\-4v\.arXiv preprint arXiv:2310\.11441\.External Links:[Link](https://arxiv.org/abs/2310.11441)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.19.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.2.1),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Yanget al\.\(2026b\)X\. Yang, L\. Li, H\. Zhou, T\. Zhu, X\. Qu, Y\. Fan, Q\. Wei, R\. Ye, L\. Kang, Y\. Qin, Z\. Kou, D\. Liu, Q\. Li, N\. Ding, S\. Chen, and J\. ShaoToward efficient agents: memory, tool learning, and planning\.arXiv preprint arXiv:2601\.14192\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2601.14192),[Link](https://arxiv.org/abs/2601.14192)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p3.1),[§2\.1](https://arxiv.org/html/2609.02309#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1)\.
- Yanget al\.\(2025a\)Y\. Yang, Y\. Wang, D\. Li, Z\. Luo, B\. Chen, C\. Huang, and J\. LiAria\-ui: visual grounding for gui instructions\.InFindings of the Association for Computational Linguistics: ACL 2025,pp\. 22418–22433\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.1152),[Link](https://aclanthology.org/2025.findings-acl.1152/)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.27.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.18.4.3),[§3\.3](https://arxiv.org/html/2609.02309#S3.SS3.p1.1)\.
- Yanget al\.\(2025b\)Y\. Yang, Z\. Yang, Z\. Dou, A\. T\. Nguyen, O\. Attia, A\. Szot, K\. You, M\. Feng, R\. Ramrakhya, A\. T\. Toshev, C\. Huang, Y\. Yang, and Z\. GanUltraCUA: scaling computer use agent through gui and programmatic control\.Note:OpenReview, ICLR 2026 desk rejected submissionExternal Links:[Link](https://openreview.net/forum?id=yVP2ldY9lw)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.24.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.5.1)\.
- Youet al\.\(2024\)K\. You, H\. Zhang, E\. Schoop, F\. Weers, A\. Swearngin, J\. Nichols, Y\. Yang, and Z\. GanFerret\-ui: grounded mobile ui understanding with multimodal llms\.arXiv preprint arXiv:2404\.05719\.External Links:[Link](https://arxiv.org/abs/2404.05719)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.11.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.17.2.2),[§3\.2](https://arxiv.org/html/2609.02309#S3.SS2.p1.1)\.
- Yuet al\.\(2026\)S\. Yu, G\. Li, W\. Shi, and P\. QiPolySkill: learning generalizable skills through polymorphic abstraction for continual learning\.Note:OpenReview, ICLR 2026 posterExternal Links:[Link](https://openreview.net/forum?id=KdEsujyiSV)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.4.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.21.2.2),[§5\.1](https://arxiv.org/html/2609.02309#S5.SS1.p1.1)\.
- Zhanget al\.\(2025a\)J\. Zhang, K\. Chen, Z\. Lu, E\. Zhou, Q\. Yu, and J\. ZhangPrune4Web: dom tree pruning programming for web agent\.arXiv preprint arXiv:2511\.21398\.External Links:[Link](https://arxiv.org/abs/2511.21398)Cited by:[Table 2](https://arxiv.org/html/2609.02309#A4.T2.2.7.1.1.1),[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.9.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.16.4.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.22.2.1),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p2.1),[§3\.1](https://arxiv.org/html/2609.02309#S3.SS1.p1.1),[§5\.2](https://arxiv.org/html/2609.02309#S5.SS2.p1.1)\.
- Zhanget al\.\(2025b\)Z\. Zhang, Y\. Lu, Y\. Fu, Y\. Huo, S\. Yang, Y\. Wu, H\. Si, X\. Cong, H\. Chen, Y\. Lin, J\. Xie, W\. Zhou, W\. Xu, Y\. Zhang, Z\. Su, Z\. Zhai, X\. Liu, Y\. Mei, J\. Xu, H\. Tian, C\. Wang, C\. Chen, Y\. Yao, Z\. Liu, and M\. SunAgentCPM\-gui: building mobile\-use agents with reinforcement fine\-tuning\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing: System Demonstrations,pp\. 155–180\.External Links:[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-demos.12),[Link](https://aclanthology.org/2025.emnlp-demos.12/)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.12.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.6.1),[§6\.1](https://arxiv.org/html/2609.02309#S6.SS1.p3.1)\.
- Zhenget al\.\(2025\)B\. Zheng, M\. Y\. Fatemi, X\. Jin, Z\. Z\. Wang, A\. Gandhi, Y\. Song, Y\. Gu, J\. Srinivasa, G\. Liu, G\. Neubig, and Y\. SuSkillWeaver: web agents can self\-improve by discovering and honing skills\.arXiv preprint arXiv:2504\.07079\.External Links:[Link](https://arxiv.org/abs/2504.07079)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.3.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.21.2.1),[§5\.1](https://arxiv.org/html/2609.02309#S5.SS1.p1.1)\.
- Zhonget al\.\(2026\)H\. Zhong, F\. Faisal, L\. França, T\. Leesatapornwongsa, A\. Szekeres, K\. Rong, and S\. NathActionEngine: from reactive to programmatic gui agents via state machine memory\.arXiv preprint arXiv:2602\.20502\.External Links:[Link](https://arxiv.org/abs/2602.20502)Cited by:[Table 4](https://arxiv.org/html/2609.02309#A4.T4.2.6.1.1.1),[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.16.1.1.1),[§1](https://arxiv.org/html/2609.02309#S1.p2.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.21.3.2),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.25.2.1),[§5\.1](https://arxiv.org/html/2609.02309#S5.SS1.p2.1),[§6\.2](https://arxiv.org/html/2609.02309#S6.SS2.p1.1)\.
- Zhouet al\.\(2026a\)B\. Zhou, Z\. Xu, W\. Li, J\. Xiao, and H\. WangEfficient long\-horizon gui agents via training\-free kv cache compression\.arXiv preprint arXiv:2603\.00188\.External Links:[Link](https://arxiv.org/abs/2603.00188)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.12.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.20.2.2),[§4\.2](https://arxiv.org/html/2609.02309#S4.SS2.p1.1)\.
- Zhouet al\.\(2026b\)J\. Zhou, J\. Wang, W\. Zhang, W\. Liu, Z\. Zhang, X\. Lou, W\. Zhang, H\. Deng, and J\. WangColorBrowserAgent: complex long\-horizon browser agent with adaptive knowledge evolution\.arXiv preprint arXiv:2601\.07262\.External Links:[Link](https://arxiv.org/abs/2601.07262)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.4.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.2.2),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Zhouet al\.\(2023\)S\. Zhou, F\. F\. Xu, H\. Zhu, X\. Zhou, R\. Lo, A\. Sridhar, X\. Cheng, T\. Ou, Y\. Bisk, D\. Fried, U\. Alon, and G\. NeubigWebArena: a realistic web environment for building autonomous agents\.arXiv preprint arXiv:2307\.13854\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.2307.13854),[Link](https://arxiv.org/abs/2307.13854)Cited by:[§1](https://arxiv.org/html/2609.02309#S1.p1.1),[§1](https://arxiv.org/html/2609.02309#S1.p5.1),[§2\.1](https://arxiv.org/html/2609.02309#S2.SS1.p1.2),[§2\.2](https://arxiv.org/html/2609.02309#S2.SS2.p1.1)\.
- Zhouet al\.\(2025a\)X\. Zhou, G\. Chen, Y\. Xie, Z\. Li, K\. Zhou, S\. Wang, S\. Yang, Z\. Tian, and R\. ShaoHiconAgent: history context\-aware policy optimization for gui agents\.arXiv preprint arXiv:2512\.01763\.External Links:[Link](https://arxiv.org/abs/2512.01763)Cited by:[Table 3](https://arxiv.org/html/2609.02309#A4.T3.2.7.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.19.3.3),[§4\.1](https://arxiv.org/html/2609.02309#S4.SS1.p1.1)\.
- Zhouet al\.\(2025b\)Y\. Zhou, S\. Dai, S\. Wang, K\. Zhou, Q\. Jia, and J\. XuGUI\-g1: understanding r1\-zero\-like training for visual grounding in gui agents\.Note:OpenReview, NeurIPS 2025 posterExternal Links:[Link](https://openreview.net/forum?id=1XLjrmKZ4p)Cited by:[Table 5](https://arxiv.org/html/2609.02309#A4.T5.2.8.1.1.1),[Figure 2](https://arxiv.org/html/2609.02309#S2.F2.pic1.24.4.2)\.

## Appendix AGUI\-agent

A single next\-action equation hides the fact that deployment cost accumulates over a full interaction\. We therefore also view a GUI\-agent episode as a system\-level trajectory\. Letsts\_\{t\}denote the latent environment state,oto\_\{t\}the GUI observation extracted from that state,mtm\_\{t\}the retained context or memory,ata\_\{t\}the executed interface action, andvtv\_\{t\}a verification or reflection signal after execution\. A compact abstraction is

ot\\displaystyle o\_\{t\}=Ω⁡\(st\),\\displaystyle=\\Omega\(s\_\{t\}\),\(2\)mt\\displaystyle m\_\{t\}=Um​\(mt−1,ot,at−1,vt−1\),\\displaystyle=U\_\{m\}\(m\_\{t\-1\},o\_\{t\},a\_\{t\-1\},v\_\{t\-1\}\),\(3\)at\\displaystyle a\_\{t\}∼πθ​\(g,ot,mt\),\\displaystyle\\sim\\pi\_\{\\theta\}\(g,o\_\{t\},m\_\{t\}\),\(4\)st\+1\\displaystyle s\_\{t\+1\}∼𝒯⁡\(st,at\),\\displaystyle\\sim\\mathcal\{T\}\(s\_\{t\},a\_\{t\}\),\(5\)vt\\displaystyle v\_\{t\}=V⁡\(g,st,ot,at,st\+1\),\\displaystyle=V\(g,s\_\{t\},o\_\{t\},a\_\{t\},s\_\{t\+1\}\),\(6\)whereΩ\\Omegacovers screenshot, DOM, HTML, AxTree, parser, or hybrid perception modules;UmU\_\{m\}covers history summarization, retrieval, compression, or KV\-cache reuse;πθ\\pi\_\{\\theta\}covers the planner, grounding model, or orchestrator;𝒯\\mathcal\{T\}is the GUI environment transition; andVVdenotes explicit verifiers, self\-reflection, post\-condition checks, or implicit task\-progress feedback\. The full execution is

τ=\(s0,o0,m0,a0,v0,…,sT,oT,mT\),\\tau=\(s\_\{0\},o\_\{0\},m\_\{0\},a\_\{0\},v\_\{0\},\\ldots,s\_\{T\},o\_\{T\},m\_\{T\}\),\(7\)with task outcomeS⁡\(τ\)∈\{0,1\}S\(\\tau\)\\in\\\{0,1\\\}or a graded completion score\. Sections[3](https://arxiv.org/html/2609.02309#S3)–[6](https://arxiv.org/html/2609.02309#S6)can be read as four ways to reduce this trajectory cost: cheaper observations, cheaper memory and context updates, fewer or more recoverable actions, and lower planner/runtime overhead\.

For ease of reading in the main text, we collect the tree\-structured taxonomy overview and the section\-wise paper summary tables in this appendix\.

## Appendix BLiterature Expansion Protocol

We expanded the seed draft subsection by subsection rather than treating literature search as a single flat retrieval problem\. For each existing subsection, we began with the papers already cited in the draft, then used targeted search over arXiv, ACL Anthology, and OpenReview together with backward and forward citation chaining to surface closely related work\. We retained papers only when they directly addressed efficiency in the relevant subsection: observation length or actionability, memory size or retrieval cost, action count or recoverability, or planner/runtime overhead\. We excluded generic GUI grounding papers, pure benchmark\-generation papers, and unrelated long\-horizon agent work unless they directly informed the subsection’s efficiency mechanism\. Where a paper had both a preprint and a later venue version, we normalized it to a single canonical entry\.

## Appendix CAdded Works by Subsection

Table 1:Additional works integrated beyond the seed draft, grouped by the existing subsection taxonomy\.
## Appendix DTaxonomy Figure and Section Summary Tables

For ease of reading in the main text, we collect the tree\-structured taxonomy overview and the section\-wise paper summary tables in this appendix\. In Tables[2](https://arxiv.org/html/2609.02309#A4.T2)–[5](https://arxiv.org/html/2609.02309#A4.T5),*NR*means that the source paper does not report a comparable quantitative efficiency metric\. Reported percentages, speedups, token counts, costs, and step counts are copied from the authors’ baseline comparisons and should be read as an evidence ledger rather than a direct meta\-analysis\.

Table 2:Observation\-efficiency papers discussed in this survey, grouped by the existing subsection structure\.PaperVenuePlatform\+UICore mechanismReported efficiency signalUnreported / shifted costCodeTextual Observation ReductionAgent\-E[Abuelsaad et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib1)arXiv’24Web/DOMDOM filtering150–220 s/task; about25 LLM calls/task; raw DOM up to800k tokens\.Filtered\-token saving and DOM\-filter latency areNR\.[GitHub](https://github.com/EmergenceAI/Agent-E)Beyond Pixels[Schiepanski and Piël \(2025\)](https://arxiv.org/html/2609.02309#bib.bib3)arXiv’25Web/DOMDOM compressionCompressed DOM snapshots stay around the10310^\{3\}tokenorder\.DOM downsampling runtime isNR\.–LineRetriever[Kerboua et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib5)arXiv’25Web/AxTreeline retrievalObservation reduced by61%/72%/73%; retrieves up to10chunks of100tokens\.Retriever latency and extra model cost areNR\.–FocusAgent[Kerboua et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib6)arXiv’25Web/AxTreeAxTree filtering\>\>50%average reduction, often\>\>80%; caps context at2k tokens\.Full wall\-clock retrieval overhead isNR\.–Prune4Web[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib7)arXiv’25Web/DOMDOM/action pruningDOMs:10k–100k tokens; example shrinks\>\>500elements to<<20\.Filter\-program generation/runtime isNR\.–Read More, Think More[Enomoto et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib41)arXiv’26Web/HTML\+Axadaptive viewWorkArena HTML pages:40k–500k tokens; look\-back settings:4/9steps\.Adaptive\-view saving and latency areNR\.–Region\-Focused Visual PerceptionSeeClick[Cheng et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib8)ACL’24GUI/screenGUI pretrainingNR; no reported token, latency, memory, or step saving\.Screenshot\-only runtime cost isNR\.[GitHub](https://github.com/njucckevin/SeeClick)Ferret\-UI[You et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib9)arXiv’24Mobile/screenscreen cropsAny\-resolution processing splits each screen into2 sub\-images\.Crop encoding and inference overhead areNR\.–R\-VLM[Park et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib54)ACL F\.’25GUI/screenregion proposalsReports about5\.6 s/sampleand up to2×\\timesinference\-latency cost\.Region proposal shifts cost to extra inference\.–RegionFocus[Luo et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib11)arXiv’25GUI/screenzoom searchAverage trajectory overhead66\.8%; step count increases by19\.74%\.Zoom\-search latency is only partially reported\.–ShowUI[Lin et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib42)CVPR’25GUI/screentoken pruningRemoves33%redundant visual tokens; reports1\.4×\\timestraining speedup\.End\-to\-end agent latency isNR\.[GitHub](https://github.com/showlab/ShowUI)DiMo\-GUI[Wu et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib55)EMNLP’25GUI/screenadaptive zoomUses up to7zoom iterations\.Extra zoom\-call latency isNR\.–ScreenSpot\-Pro[Li et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib43)arXiv’25GUI/high\-rescascaded searchBenchmark screens often exceed3k×\\times2kresolution\.Benchmark only; runtime metric isNR\.–SimpAgent[Chen et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib44)ICCV’25GUI/screen\+histscreen maskingHistory image:64 tokens; action output:10–20 tokens; LLM branch:27%FLOP reduction\.Simplifier overhead and full\-task latency areNR\.–Observation Enrichment via Parsing and HybridizationSet\-of\-Mark[Yang et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib12)arXiv’23GUI/screenregion marksNR; no comparable token, latency, or memory metric\.Mark rendering cost isNR\.–ScreenAI[Baechler et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib13)arXiv’24GUI/screen\-textscreen captioningReports670M/2B/5Bmodel scales and up to8122input resolution\.GUI\-agent latency or token saving isNR\.–OmniParser[Lu et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib14)arXiv’24GUI/parserscreen parsingAdds2parser models; one setting keeps top\-50relevant elements\.Parser latency and extra calls areNR\.[GitHub](https://github.com/microsoft/OmniParser)Tree\-of\-Lens[Fan et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib10)arXiv’24GUI/pointedlayout treeNR; no reported token, latency, or parser\-cost saving\.Detector/layout\-tree runtime isNR\.[GitHub](https://github.com/eric-ai-lab/Screen-Point-and-Read)GUI\-Actor[Wu et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib56)NeurIPS’25GUI/screenregion actionsAction head adds about20Mparams for 2B and100Mfor 7B; candidates inone forward pass\.Wall\-clock latency isNR\.[GitHub](https://github.com/microsoft/GUI-Actor)Agent\-S[Agashe et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib15)arXiv’24OS/img\+AXhybrid perceptionNR; no observation\-side cost metric\.Step count and wall\-clock overhead areNR\.[GitHub](https://github.com/simular-ai/Agent-S)Ferret\-UI 2[Li et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib16)arXiv’24Cross\-platformadaptive scalingNR; no runtime or token\-efficiency metric\.Adaptive\-scaling overhead isNR\.–UGround[Gou et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib17)ICLR’25GUI/screenvisual groundingAdaptive resolution uses about2/3of fixed 1344×\\times1344 visual tokens\.End\-to\-end agent latency isNR\.[GitHub](https://github.com/OSU-NLP-Group/UGround)Aria\-UI[Yang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib19)ACL F\.’25GUI/screeninstruction grounding3\.9Bactivated params; phase\-1 training: about18 h/10ksteps\.Exact inference latency isNR\.–Aguvis[Xu et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib18)arXiv’24GUI/screenpure\-vision loopText agents:4k–6k tokens/step; pure\-vision loop:1,196 tokens;70%input\-token reduction\.Vision\-encoder latency isNR\.[GitHub](https://github.com/xlang-ai/aguvis)UI\-TARS[Qin et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib20)arXiv’25GUI/screennative screen agentModel scales:2B/7B/72B; training budget about50B tokens; OSWorld budgets:15/50steps\.Training cost and inference latency areNR\.[GitHub](https://github.com/bytedance/UI-TARS)Table 3:Context\- and memory\-efficiency papers discussed in this survey, grouped by the existing subsection structure\.PaperVenuePlatform\+UICore mechanismReported efficiency signalUnreported / shifted costCodeSummary\-Based Compression and Selective Look\-BackAgent\-S[Agashe et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib15)arXiv’24OS/img\+AXepisodic memoryNR; no memory footprint, prompt\-growth, or latency metric\.Memory update/storage overhead isNR\.[GitHub](https://github.com/simular-ai/Agent-S)ColorBrowserAgent[Zhou et al\. \(2026b\)](https://arxiv.org/html/2609.02309#bib.bib45)arXiv’26Web/GUIprogress summariesInteraction horizon capped at30 steps\.Summary\-token saving and latency areNR\.–GUI\-Rise[Liu et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib57)NeurIPS’25GUI/imgtask summariesNR; no summary compression ratio or retrieval\-cost metric\.Summary and retrieval overhead areNR\.[GitHub](https://github.com/Leon022/GUI-Rise-code)PAL\-UI[Liu et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib21)arXiv’25GUI/imgsummary \+ retrievalNR; no memory footprint or latency metric\.Retrieval cost isNR\.–HiconAgent[Zhou et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib22)arXiv’25GUI/imgcontext samplingCompressed history uses25\.21T FLOPsvs\.35\.75Tuncompressed 3B and62\.31T7B; reports60%FLOP reduction\.Wall\-clock latency and measured memory footprint areNR\.[GitHub](https://github.com/iLearn-Lab/CVPR26-HiconAgent)SimpAgent[Chen et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib44)ICCV’25GUI/img\+histhistory pruningHistory image:64 tokens; LLM branch:27%FLOP reduction\.Simplifier overhead and full\-task latency areNR\.–Read More, Think More[Enomoto et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib41)arXiv’26Web/historyhistory diffsHTML pages:40k–500k tokens; history settings:4/9past steps\.History\-diff compression ratio and latency areNR\.–Runtime\-Level Representation CompressionGUI\-KV[Huang et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib23)arXiv’25GUI/VLM cacheKV pruning38\.9%fewer MFLOPs/decoded token at 5 screenshots;5–20%cache budgets; 5 screenshots can exceed80GBGPU memory\.Prefill and end\-to\-end latency remain partially unresolved\.–ST\-Lite[Zhou et al\. \(2026a\)](https://arxiv.org/html/2609.02309#bib.bib25)arXiv’26GUI/VLM cachecache pruning10–20%cache budgets;2\.45×\\timesdecoding speedup;1\.40×\\timesend\-to\-end speedup; prefill about1\.0×\\times\.Speedup is mostly decode\-side\.–Continuous Memory[Wu et al\. \(2025d\)](https://arxiv.org/html/2609.02309#bib.bib24)arXiv’25GUI/latent mem\.latent memoryEach trajectory becomes8 embeddings; raw trajectories can exceed15k tokens; collection cost about$4k; tunes1\.2%params\.Latent retrieval and embedding overhead areNR\.[GitHub](https://github.com/WenyiWU0111/CoMEM-Agent)SecAgent[Xie et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib26)arXiv’26Mobile/sem\. ctxsemantic summariesN=1 to N=5: ITC2,239→\\rightarrow3,642\(\+62\.7%\); TTFT0\.11→\\rightarrow0\.22; TPS140→\\rightarrow122\.Full\-task runtime and training cost are not unified\.–Table 4:Action\-efficiency papers discussed in this survey, grouped by the existing subsection structure\.PaperVenuePlatform\+UICore mechanismReported efficiency signalUnreported / shifted costCodeAction AbstractionSkillWeaver[Zheng et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib27)arXiv’25Web/APIsskill APIsNR; no reused\-skill action\-count or latency saving\.API overhead isNR\.[GitHub](https://github.com/OSU-NLP-Group/SkillWeaver)PolySkill[Yu et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib58)ICLR’26Web/skillspolymorphic skillsLearned functions typically cover2–5 GUI steps\.Exact step reduction and skill\-search cost areNR\.–Mobile\-Agent\-E[Wang et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib28)arXiv’25Mobile/skillsshortcut skillsNR; no shortcut action\-count or latency saving\.Shortcut discovery/runtime cost isNR\.[GitHub](https://github.com/X-PLUG/MobileAgent)ActionEngine[Zhong et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib38)arXiv’26GUI/programscompiled routinesCost$0\.71→\\rightarrow$0\.06\(11\.83×\\times\); latency237\.5→\\rightarrow118\.3 s; input62\.3k→\\rightarrow8\.1k tokens; calls10\.2→\\rightarrow1\.8\.Compiler, validator, and first\-recovery latency remain shifted costs\.–CoAct\-1[Song et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib53)ICLR’26OS/GUI\+codecode executionAverage steps:10\.15vs\. GTA\-115\.22and UI\-TARS14\.90\.Code\-execution overhead and sandboxing cost areNR\.–Action Pruning, Verification, and RecoveryPrune4Web[Zhang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib7)arXiv’25Web/DOMcandidate pruningExample candidate set shrinks from\>\>500DOM elements to<<20\.Pruner generation/runtime latency isNR\.–V\-Droid[Dai et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib29)arXiv’25Mobile/imgaction scoringInput2\.6k–8\.9k tokens;0\.7 s/decision;4\.3 s/step; typical agents reported above20 s/step\.Device/controller overhead is not separated\.–VeriSafe Agent[Lee et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib47)arXiv’25Mobile/imgintent checksRecovery examples need1–3 actions; tasks above10 stepscan exceed$1\.Verifier latency and cost curves are not tabulated\.–GUI\-Shepherd[Chen et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib60)OpenRev’25GUI/processreward verifierNR; no verifier\-call, latency, or cost metric\.Extra verifier calls and wall\-clock latency areNR\.–SenseAct[Cai et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib59)ICLR Wk\.’26GUI/typed actstyped checksUI exposure reduced by65\.51%\.Typed\-check and recovery latency areNR\.–BacktrackAgent[Wu et al\. \(2025c\)](https://arxiv.org/html/2609.02309#bib.bib30)EMNLP’25Mobile/imgbacktrackingSpeed ratios0\.451/0\.517/0\.482; action execution about0\.25 s\.Recovery shifts cost to slower execution\.–LongHorizonUI[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67)ICLR’26OS/img\+idxrollback loopAverage trajectory24\.6 steps, max37; verifier choice changes latency8\.26 s→\\rightarrow5\.74–6\.59 s/step\.Rollback verifier\-call overhead remains\.–Exploration ControlLASER[Ma et al\. \(2023\)](https://arxiv.org/html/2609.02309#bib.bib31)arXiv’23Web/textstate searchNR; no search\-cost, latency, or action\-count saving\.Exploration overhead isNR\.–Auto\-Intent[Kim et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib32)arXiv’24Web/demo\-guidedintent guidanceTraining uses about1k A100\-40GB GPU hoursplus0\.4kGPU\-hour search; inference up to5 tokens\.Action\-count saving isNR\.–OpenWebVoyager[He et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib33)arXiv’24Web/mmexplore\-optimizeSamples each task trajectory up to5times; keeps at most3 screenshotsin context\.Exploration amortization, latency, and cost areNR\.–GUI\-explorer[Xie et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib46)ACL’25GUI/mixedtransition mining66 sper interaction step; ranker28\.5 s; mining is42\.9%of runtime\.Transition mining is a major overhead\.–WebOperator[Dihan et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib61)OpenRev’25Web/searchtree searchTree search uses31\.34 actionsvs\.24\.79naive \(\+6\.55\); up to3candidates/step\.Backtracking increases action count\.–MobileUse[Li et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib37)OpenRev’25Mobile/imgproactive exploreExploration cap:100 steps/appat19\.5 s/step; reflection overhead reducible to10%\.Exploration cost is amortized, not eliminated\.–Table 5:Planner\-side and system\-efficiency papers discussed in this survey, grouped by the existing subsection structure\.PaperVenuePlatform\+UICore mechanismReported efficiency signalUnreported / shifted costCodePlanner\-Side EfficiencyAndroidWorld[Rawles et al\. \(2024\)](https://arxiv.org/html/2609.02309#bib.bib34)arXiv’24Mobile/evalrealistic evalBenchmark footprint about2GB memoryand8GB disk;116tasks across20apps\.Benchmark only; agent latency/cost metric isNR\.–MMBench\-GUI[Wang et al\. \(2025a\)](https://arxiv.org/html/2609.02309#bib.bib48)arXiv’25Cross\-platform evalefficiency eval50\-stepbudget; redundant\-step cost EQ2 is7–8; privacy noise40%; specialist cost16%\.Evaluation metrics expose cost but do not optimize runtime\.–OSWorld\-Human[Abhyankar et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib51)arXiv’25OS/human evalhuman profilingAgents need2\.7×\\times–4\.3×\\timesmore steps; reflection takes76%–96%of task latency\.Profiling only; no new optimization\.–UI\-R1[Lu et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib35)arXiv’25GUI action pred\.RL policyRFT uses only136training samples; training takes about8 hon8 RTX 4090GPUs\.Inference latency and output\-token cost areNR; high pixel settings can cause OOM\.–Think Twice, Click Once[Tang et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib36)arXiv’25GUI groundingadaptive reasoningAdaptive slow thinking raises processing time2\.6→\\rightarrow5\.4 s; data include300kexamples and150kslow samples\.Higher reasoning depth trades for about2×\\timeslatency\.–GUI\-G1[Zhou et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib62)OpenRev’25GUI groundingfast\-thinking RLOutput tokens:37/39/39on mobile/desktop/web vs\.107/107/114for InfiGUI\-R1, about1/3\.Wall\-clock latency isNR\.–AdaGUI\-R1[Chen et al\. \(2025c\)](https://arxiv.org/html/2609.02309#bib.bib63)OpenRev’25Mobile/GUIreasoning scheduleUnnecessary reasoning tokens reduced by40%; harder\-example schedule adds23\.5%FLOPs\.Latency and end\-to\-end cost areNR\.–MobileUse[Li et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib37)OpenRev’25Mobile/imgon\-demand reflectionReflection overhead can be reduced to10%; exploration costs up to100 steps/appat19\.5 s/step\.Exploration cost is amortized across tasks\.–MobileWizard[Lin et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib49)OpenRev’25Mobile/imgstructured planningUses24\.5kpublic trajectories plus300remedial trajectories; fewer than50ktrajectories\.Planner runtime, latency, and memory cost areNR\.–AgentCPM\-GUI[Zhang et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib50)EMNLP SD\.’25Mobile/imgcompact actions470katomic steps, about8\.5 steps/trajectory; retains last4actions/images; max\_new\_tokens2048\.Actual output\-token saving and latency areNR\.[GitHub](https://github.com/OpenBMB/AgentCPM-GUI)Agent S2[Agashe et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib52)COLM’25OS/img\+modsplanner splitNR; no specialist\-routing latency, cost, or memory metric\.Specialist orchestration overhead isNR\.[GitHub](https://github.com/simular-ai/Agent-S)InfiGUIAgent[Liu et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib64)EACL’26GUI/mmcompact reflectionTraining uses8 A800\-80GBGPUs and32kcontext\.Exact inference latency and memory saving areNR\.–System Efficiency Beyond the PlannerActionEngine[Zhong et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib38)arXiv’26GUI/programsruntime programsCost$0\.71→\\rightarrow$0\.06\(11\.83×\\times\); latency237\.5→\\rightarrow118\.3 s; input62\.3k→\\rightarrow8\.1k tokens; calls10\.2→\\rightarrow1\.8\.Compiler, validator, sandbox, and first\-recovery overhead remain shifted costs\.–CORE[Fan et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib39)NeurIPS’25Mobile/cloudlocal\-cloud routingUI exposure reduced by55\.60%/34\.96%; latency1\.52–1\.66×\\timesbaseline; cloud tokens0\.94–1\.15×\\times\.Privacy saving trades for local\-routing latency\.–GUIGuard[Wang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib40)arXiv’26GUI/privacyprivacy stagesNR; no privacy\-detector runtime or deployment\-overhead metric\.Detector/runtime overhead isNR\.–CoAct\-1[Song et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib53)ICLR’26OS/GUI\+codebackend routingAverage steps:10\.15vs\. GUI baselines around15steps\.Code backend overhead and safety cost areNR\.–Agent S2[Agashe et al\. \(2025\)](https://arxiv.org/html/2609.02309#bib.bib52)COLM’25OS/modularsplit modulesNR; no module\-routing latency, cost, or memory metric\.Module orchestration overhead isNR\.[GitHub](https://github.com/simular-ai/Agent-S)IntentCUA[Lee et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib65)AAMAS’26OS/multi\-agentplan reuseNR; public source PDF was unavailable in this extraction pass\.Plan\-reuse overhead isNR\.–OS\-Symphony[Yang et al\. \(2026a\)](https://arxiv.org/html/2609.02309#bib.bib68)arXiv’26OS/frameworkorch\. runtimeNR; no message\-protocol or orchestration\-latency metric\.Framework overhead isNR\.–LongHorizonUI[Kang et al\. \(2026\)](https://arxiv.org/html/2609.02309#bib.bib67)ICLR’26OS/reflectionrollback runtimeAverage trajectory24\.6 steps, max37; verifier choice changes latency8\.26 s→\\rightarrow5\.74–6\.59 s/step\.Rollback verifier\-call overhead remains\.–UltraCUA[Yang et al\. \(2025b\)](https://arxiv.org/html/2609.02309#bib.bib66)OpenRev’25GUI\+prog\.GUI/API fusionReports11%fewer steps overall; hybrid control uses14\.9%fewer steps\.Tool/API routing latency and runtime cost areNR\.–