Paper page - UI-KOBE: Knowledge-Oriented Behavior Exploration for Lightweight Graph-Guided GUI Agents

Source: https://huggingface.co/papers/2605.29534

Abstract

UI-KOBE framework enhances lightweight mobile GUI agents by incorporating reusable app-specific graph knowledge to improve task planning and execution efficiency.

Recent advances inmobile GUI agentshave shown strong potential for automating mobile tasks, but most effective systems still depend on largevision-language modelsfor screenshot understanding and long-horizon planning. Small GUI agents that can be deployed directly on mobile devices are more attractive for practical use, offering lower inference cost and better protection of sensitive on-device information. However, due to limited model capacity, such lightweight agents remain unreliable when planning and executing GUI tasks end-to-end from screenshots alone. We propose Knowledge-Oriented Behavior Exploration (UI-KOBE), a framework that improves lightweightmobile GUI agentswith reusable app-specific graph knowledge. UI-KOBE first autonomously explores a mobile application and constructs anapp knowledge graph, where nodes represent distinctUI statesand edges representexecutable transitions. At runtime, a lightweight GUI agent uses the graph as external guidance: given a user task and the current screenshot, it identifies the current graph node and selects amongself-loop actions,neighboring transitions,task completion, orfallback free actionsassociated with that node. By supportingruntime decisionswith app-specific graph guidance, UI-KOBE reduces the burden ofend-to-end GUI planningand helps lightweight models perform mobile GUI tasks more effectively, offering a practical step toward efficient, interpretable, and privacy-conscious on-device GUI agents.

View arXiv pageView PDFAdd to collection

Get this paper in your agent:

hf papers read 2605\.29534

Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash

Models citing this paper0

No model linking this paper

Cite arxiv.org/abs/2605.29534 in a model README.md to link it from this page.

Datasets citing this paper0

No dataset linking this paper

Cite arxiv.org/abs/2605.29534 in a dataset README.md to link it from this page.

Spaces citing this paper0

No Space linking this paper

Cite arxiv.org/abs/2605.29534 in a Space README.md to link it from this page.

Collections including this paper0

No Collection including this paper

Add this paper to acollectionto link it from this page.