MAGA: Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation

arXiv cs.AI 论文

摘要

This paper introduces Maga, a method for consolidating domain-specific GUI agents into a single cross-platform policy via structured action distillation, reallocating training signals to focus on erroneous actions. It achieves strong success rates across mobile, web, and desktop benchmarks.

arXiv:2607.29320v1 Announce Type: new Abstract: Graphical user interface (GUI) agents based on large language models are increasingly deployed across mobile, web, and desktop environments. However, existing agents are typically domain-specific, limiting the deployment and user experience. This motivates the consolidation of specialized models into a single cross-environment policy. Weight merging directly merges domain-specific experts but can corrupt executable actions under expert disagreement, while on-policy distillation (OPD) avoids conflicting teacher supervision yet still treats all response tokens equally during distillation, ignoring that action tokens are the only interface between the environment and the agent. To address this, We introduce MAGA that re-allocates training signal according to the structured action. Based on the correctness of the generated action, it suppresses unnecessary or invalid distillation signals and focuses learning on erroneous actions. Besides, a training-only hint optimizes the supervision signal provided by domain-specific teachers without changing the student input. Across two model scales, MAGA achieves the highest mean success rate, outperforming the strongest baseline by 2.0% at 8B and achieves almost the same average performance with teachers.
查看原文
查看缓存全文

缓存时间: 2026/08/03 07:31

# Multi-Platform Self-Fusion of GUI Agents via Structured Action Distillation
Source: [https://arxiv.org/html/2607.29320](https://arxiv.org/html/2607.29320)
Hang Yan1,2Zhangxuan Gu2\\correspondingBeitong Zhou2Jiaxuan Chen2,3Runze Li2Yusong Hu2 Shuheng Shen2Changhua Meng2

###### Abstract

Graphical user interface \(GUI\) agents based on large language models are increasingly deployed across mobile, web, and desktop environments\. However, existing agents are typically domain\-specific, limiting the deployment and user experience\. This motivates the consolidation of specialized models into a single cross\-environment policy\. Weight merging directly merges domain\-specific experts but can corrupt executable actions under expert disagreement, while on\-policy distillation \(OPD\) avoids conflicting teacher supervision yet still treats all response tokens equally during distillation, ignoring that action tokens are the only interface between the environment and the agent\. To address this, We introduceMagathat re\-allocates training signal according to the structured action\. Based on the correctness of the generated action, it suppresses unnecessary or invalid distillation signals and focuses learning on erroneous actions\. Besides, a training\-only hint optimizes the supervision signal provided by domain\-specific teachers without changing the student input\. Across two model scales,Magaachieves the highest mean success rate, outperforming the strongest baseline by2\.0%2\.0\\%at 8B and achieves almost the same average performance with teachers\.

## 1Introduction

![Refer to caption](https://arxiv.org/html/2607.29320v1/x1.png)Figure 1:Results from three benchmarks, including MobileWorld \(MW\), OSWorld \(OSW\), and WebVoyager \(WV\)\. \(a\) On 900 random samples, we identify 66 tasks where domain\-specific models exhibited high disagreement, and weight merging reduces success rate of the merged model by 10%–24%\. \(b\) Success rate of models based on Qwen3\-VL\-8B across three benchmarks\. The horizontal dashed lines mark the domain\-specific teachers\.Magais the strongest unified model on every benchmark and exceeds the corresponding domain\-specific teacher on OSWorld and WebVoyager\.Recent advances in vision\-language models \(VLMs\)\(Zhu et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib45); Li et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib12); Zhang et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib38); Liang et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib14); Zhang et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib39)\)have improved visual understanding, multimodal reasoning, and grounded decision making\. These capabilities support graphical user interface \(GUI\) agents\(Zhou et al\.[2025b](https://arxiv.org/html/2607.29320#bib.bib44); Xu et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib30); Team et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib22); Tang et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib21)\)that interpret screen content and execute actions from natural language instructions\. Earlier GUI agents typically focused on mobile applications\(Sun et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib19); Cheng et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib1); Gong et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib3)\), web browsers\(Yao et al\.[2022](https://arxiv.org/html/2607.29320#bib.bib37); Li et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib10); Zhang et al\.[2026d](https://arxiv.org/html/2607.29320#bib.bib41)\), or desktop operating systems\(Liu et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib15); Jia et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib7); Xue et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib31)\)separately\. These approaches introduce deployment complexity and limit cross\-domain user experience, so recent systems combine several of these domains into a single agent\.

Previous methods combine abilities across domains either by merging expert parameters\. Weight merging combines separately trained domain\-specific experts, as in Model Soup\(Wortsman et al\.[2022](https://arxiv.org/html/2607.29320#bib.bib23)\)and TIES\(Yadav et al\.[2023](https://arxiv.org/html/2607.29320#bib.bib32)\)\. GUI domains share action types such asClickandScroll, but their domain\-specific models can disagree on the corresponding decisions\. These disagreements can degrade the merged model\. As shown in Figure[1](https://arxiv.org/html/2607.29320#S1.F1)\(a\), when evaluated on samples with high disagreement among domain\-specific models, the action success rate of the merged model drops by10%10\\%–24%24\\%compared to the individual models\.

Alternatively, per\-sample routed on\-policy distillation \(OPD\)\(Lu and Lab[2025](https://arxiv.org/html/2607.29320#bib.bib16); Xiao et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib27); Yang et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib35); Wu et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib24)\)transfers each expert’s supervision to the student without combining their parameters\. For each student\-generated response, only the corresponding frozen domain\-specific model, which serves as the teacher, scores the sampled tokens\. However, ordinary OPD treats the response as a flat sequence and assigns equal weight to all tokens\. This allocation is poorly matched to GUI scenarios, where the final short structured action is the only part executed by the agent\. Moreover, GUI actions directly affect the interface state, and incorrect actions can significantly shift the system away from its intended state\.

To address this, we calibrate the detailed distillation signal during training for GUI actions\. A GUI action comprises an action type and its associated parameters\(if required\)\. Uniformly increasing supervision across all action tokens ignores this structure\. Consequently, we introduceMaga, a distillation method that allocates additional training supervision according to the structure of the action\.It operates on both the student and teacher sides\.On the student side,Magaremoves fully correct responses from distillation and focuses extra supervision on erroneous action components\. It amplifies the full action span when the action type is correct\. Otherwise, it amplifies only the type and masks the incompatible parameters\.On the teacher side, we condition the domain\-specific teacher on a hint of the correct action during training\. The hint therefore provides a more reliable signal when the teacher scores the student\-sampled tokens\. The student never receives this hint, so its input and output remains unchanged\.

We evaluateMagaon MobileWorld, OSWorld, and WebVoyager at two model scales\. As shown in Figure[1](https://arxiv.org/html/2607.29320#S1.F1)\(b\), at 8B it achieves a mean success rate of51\.2%51\.2\\%, exceeding the strongest baseline by2\.0%2\.0\\%and achieves almost the same average performance with teachers\. Our contributions can be concluded as follows:

- •We identify two limitations of existing approaches to unifying domain\-specific GUI agents: weight merging degrades when the experts disagree, and ordinary OPD under\-allocates signal to short structured actions\.
- •We introduceMaga, which re\-allocates distillation signal according to the action structure through student\-side conditional training signal re\-allocation and a training\-only teacher hint\.
- •Across three GUI domains and two model scales,Magaachieves the highest mean SR among unified methods\. At 8B, it exceeds the strongest baseline by2\.0%2\.0\\%and achieves almost the same average performance with teachers\.

## 2Related Work

##### GUI agents\.

GUI agents now operate across increasingly diverse interfaces and tasks\. Browser agents\(Yao et al\.[2022](https://arxiv.org/html/2607.29320#bib.bib37); Zheng et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib42); Zhang et al\.[2026d](https://arxiv.org/html/2607.29320#bib.bib41); Li et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib10)\)perceive and act on changing websites\. Moving to mobile devices\(Sun et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib19); Tang et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib20); Gong et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib3); Cheng et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib1)\)introduces persistent app state, cross\-app dependencies, and longer action sequences\. Desktop control agents\(Jia et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib7); Liu et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib15); Yang et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib34); Xue et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib31)\)operate over applications, files, menus, and system tools\. Despite these interface differences, practical deployment benefits from generalist agents\(Cheng et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib2); Zhou et al\.[2025a](https://arxiv.org/html/2607.29320#bib.bib43); Gu et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib4); Wu et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib26); Hu et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib6); Team et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib22); Xu et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib30)\)that cover several interface families within one model\. Following this line of work, we study how to combine separately trained per\-domain GUI models into a single agent\.

##### Model merging\.

Training a single model on mixed\-domain trajectories can improve one domain at the expense of another\(Xiao et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib27); Zhang et al\.[2026c](https://arxiv.org/html/2607.29320#bib.bib40); Xu et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib29)\), including under supervised fine\-tuning \(SFT\)\(Luong et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib17)\)or reinforcement learning \(RL\)\(Shao et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib18)\)\. Post\-hoc merging instead combines specialized checkpoints, as in Weight Soup\(Wortsman et al\.[2022](https://arxiv.org/html/2607.29320#bib.bib23)\)and TIES\(Yadav et al\.[2023](https://arxiv.org/html/2607.29320#bib.bib32)\)\. For GUI agents, domain\-specific teachers share a structured action space of action types and optional parameters\. Merging is largely harmless when they agree, but parameter disagreements can shift the prediction away from the corresponding domain\-specific teacher \(Figure[1](https://arxiv.org/html/2607.29320#S1.F1)\(a\)\)\. OPD\(Wu et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib25); Yang et al\.[2026c](https://arxiv.org/html/2607.29320#bib.bib36); Xiao et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib27); Xu et al\.[2026a](https://arxiv.org/html/2607.29320#bib.bib29)\)avoids this conflict by scoring each student\-generated token with that teacher\. However, existing OPD method\(Lian et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib13)\)distributes the original token\-level signal across long reasoning traces and a few action tokens, leaving the latter under\-supervised\. Existing evaluation also covers only two domains, leaving broader scalability untested\. We therefore introduceMagato allocate signal according to the structured action and generalize it to three GUI domains\.

![Refer to caption](https://arxiv.org/html/2607.29320v1/x2.png)Figure 2:Comparison of three strategies for unifying domain\-specific GUI agents\. \(a\) Parameter merging combines the weights of domain\-specific teachers, which can alter executable decisions when the teachers disagree\. \(b\) Per\-sample routed on\-policy distillation \(OPD\) trains a student through token\-level teacher supervision, but still treats the response as a flat token sequence, leaving the action tokens under\-supervised\. \(c\)Magaallocates distillation signal according to the action grammar\. On the student side, it removes fully correct responses from distillation\. For an incorrect response with the correct action type, it amplifies the full action span\. When the type is wrong, it amplifies only the type token and masks the downstream parameters\. On the teacher side, a training\-only hint improves supervision signal without changing the student input\.

## 3Method

This section first introduces the GUI agent interface and routed OPD \(Figure[2](https://arxiv.org/html/2607.29320#S2.F2)\(b\)\)\. We then presentMaga\(Figure[2](https://arxiv.org/html/2607.29320#S2.F2)\(c\)\), which combinesstudent\-side conditional signal allocation\(Section[3\.2](https://arxiv.org/html/2607.29320#S3.SS2)\) with ateacher\-side hint\(Section[3\.3](https://arxiv.org/html/2607.29320#S3.SS3)\) to provide targeted supervision for short, structured action outputs\.

### 3\.1Preliminaries

#### GUI Agent Interface

During each interaction step, a GUI agent receives a task instructionuu, the current visual observationoo, and an optional interaction historyHH\. We write the resulting model input asx=\(u,o,H\)x=\(u,o,H\)\. For each model call, the agent produces a responsey=\(r,a\)y=\(r,a\)containing reasoningrrand one executable actionaa\.

The action is structured asa=\(z,pz\)a=\(z,p\_\{z\}\), wherezzis a discrete action type andpzp\_\{z\}contains the parameters required by that type\. The action typezzselects the parameter schemapzp\_\{z\}, which may be empty\. These parameters may be screen coordinates, text, a URL, or a key combination\. Coordinates are normalized to the shared\[0,1000\]\[0,1000\]interface, which are one parameter class used by spatial actions\.

#### Routed On\-Policy Distillation

Letdddenote a GUI domain,x∼𝒟dx\\sim\\mathcal\{D\}\_\{d\}an input, andTdT\_\{d\}a frozen domain\-specific teacher\. For a studentπθ\\pi\_\{\\theta\}generating tokensy=\(y1,…,y\|y\|\)y=\(y\_\{1\},\\ldots,y\_\{\|y\|\}\)with prefixy<ty\_\{<t\}, calculating the exact reverse KL divergence againstTdT\_\{d\}requires summing over the full vocabulary\. For efficiency, we approximate this using only the single sampled tokenyty\_\{t\}following\(Li et al\.[2026b](https://arxiv.org/html/2607.29320#bib.bib11)\)\. We define the token\-level distillation advantage as the gradient\-stopped difference in log probabilities:

A^tOPD=sg⁡\[log⁡πTd​\(yt∣x,y<t\)−log⁡πθ​\(yt∣x,y<t\)\]\.\\widehat\{A\}\_\{t\}^\{\\mathrm\{OPD\}\}=\\operatorname\{sg\}\\\!\\left\[\\log\\pi\_\{T\_\{d\}\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\-\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\\right\]\.\(1\)Routed OPD then optimizes the student using this advantage\-weighted objective:

ℒOPD​\(θ\)=−𝔼d,x,y​\[1\|y\|​∑t=1\|y\|A^tOPD​log⁡πθ​\(yt∣x,y<t\)\]\.\\mathcal\{L\}\_\{\\mathrm\{OPD\}\}\(\\theta\)=\-\\mathbb\{E\}\_\{d,x,y\}\\Bigg\[\\frac\{1\}\{\|y\|\}\\sum\_\{t=1\}^\{\|y\|\}\\widehat\{A\}\_\{t\}^\{\\mathrm\{OPD\}\}\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\\Bigg\]\.\(2\)

### 3\.2Student\-Side Conditional Training Signal Re\-allocation

A GUI action is represented asa=\(z,pz\)a=\(z,p\_\{z\}\), where the action typezzselects a type\-specific parameter schemapzp\_\{z\}\. For example, coordinates, text, URLs, and key combinations are different parameter classes within this structure\. Letℐreason\\mathcal\{I\}\_\{\\mathrm\{reason\}\},ℐtype\\mathcal\{I\}\_\{\\mathrm\{type\}\}, andℐparam\\mathcal\{I\}\_\{\\mathrm\{param\}\}denote the reasoning, action\-type, and parameter token positions, respectively\. This decomposition yields three training cases, and we use a token weightwtw\_\{t\}to scale the routed\-OPD advantage at token positiontt\.

Training Rule 1: Correct action type, correct parameters\.When both action typezzand parameterpzp\_\{z\}are completely correct, the rollout receives the maximum reward\. We remove the whole trajectory from distillation by setting its advantage to zero:

wt=\{0,t∈ℐreason,0,t∈ℐtype,0,t∈ℐparam\.w\_\{t\}=\\begin\{cases\}0,&t\\in\\mathcal\{I\}\_\{\\mathrm\{reason\}\},\\\\ 0,&t\\in\\mathcal\{I\}\_\{\\mathrm\{type\}\},\\\\ 0,&t\\in\\mathcal\{I\}\_\{\\mathrm\{param\}\}\.\\end\{cases\}Thus, reasoning, action\-type, and parameter tokens receive no distillation signal\. The filter avoids further optimization of an action that the evaluator already accepts completely\.

Training Rule 2: Correct action type, incorrect parameters\.Unlike a fully successful rollout, this sample contains a remaining error that the teacher can correct\. We assign

wt=\{1,t∈ℐreason,1\+β,t∈ℐtype,1\+β,t∈ℐparam\.w\_\{t\}=\\begin\{cases\}1,&t\\in\\mathcal\{I\}\_\{\\mathrm\{reason\}\},\\\\ 1\+\\beta,&t\\in\\mathcal\{I\}\_\{\\mathrm\{type\}\},\\\\ 1\+\\beta,&t\\in\\mathcal\{I\}\_\{\\mathrm\{param\}\}\.\\end\{cases\}Reasoning tokens retain their ordinary routed\-OPD weight, while the complete action span, including both type and parameters, is amplified by1\+β1\+\\beta\. This prevents the training signal of the action tokens from being diluted by the longer response\.

Training Rule 3: Incorrect action type \(regardless of parameters\)\.When the action type is incorrect, the discrete type decision is the primary error to correct\. Moreover, the predicted type selects the wrong parameter schema, so its downstream parameters are not semantically valid supervision\. We therefore amplify the type tokens and mask the parameter tokens\. For a rollout with an incorrect action type, we assign token weightwtw\_\{t\}as follows:

wt=\{1,t∈ℐreason,1\+β,t∈ℐtype,0,t∈ℐparam\.w\_\{t\}=\\begin\{cases\}1,&t\\in\\mathcal\{I\}\_\{\\mathrm\{reason\}\},\\\\ 1\+\\beta,&t\\in\\mathcal\{I\}\_\{\\mathrm\{type\}\},\\\\ 0,&t\\in\\mathcal\{I\}\_\{\\mathrm\{param\}\}\.\\end\{cases\}Thus, reasoning tokens retain their original signal, the incorrect action type receives additional correction, and downstream parameters receive no distillation signal\.

### 3\.3Teacher\-Side Hint

On the teacher side, we modify only the input prompt of the routed domain\-specific teacher\. Let𝒫​\(x\)\\mathcal\{P\}\(x\)denote the standard prompt constructed from the complete inputxx, including the system instruction, interaction history, screenshot, and current task\. Leth​\(z∗\)h\(z^\{\*\}\)be a hint containing only the reference action type, and let⊕\\oplusappend this hint to the end of the final user message\. The teacher and student prompts are

𝒫T​\(x,z∗\)\\displaystyle\\mathcal\{P\}\_\{T\}\(x,z^\{\*\}\)=𝒫​\(x⊕h​\(z∗\)\),\\displaystyle=\\mathcal\{P\}\\\!\\left\(x\\oplus h\(z^\{\*\}\)\\right\),𝒫S​\(x\)\\displaystyle\\mathcal\{P\}\_\{S\}\(x\)=𝒫​\(x\)\.\\displaystyle=\\mathcal\{P\}\(x\)\.Thus, the teacher prompt differs from the student prompt only by the appended hint\. All other input content remains identical\. The hint contains no coordinates, text, URLs, keys, or reasoning and is never added to the student prompt\.

The student rollout remains unchanged\. The teacher does not decode a response and only scores the tokens sampled by the student\. This changes the teacher term in the distillation advantage but does not alter the student\-side weightwtw\_\{t\}assigned above\. The updated advantage for tokenyty\_\{t\}is

A^thint\\displaystyle\\widehat\{A\}\_\{t\}^\{\\mathrm\{hint\}\}=sg\[logπTd\(yt∣𝒫T\(x,z∗\),y<t\)\\displaystyle=\\operatorname\{sg\}\\\!\\Big\[\\log\\pi\_\{T\_\{d\}\}\\\!\\left\(y\_\{t\}\\mid\\mathcal\{P\}\_\{T\}\(x,z^\{\*\}\),y\_\{<t\}\\right\)\(3\)−logπθ\(yt∣𝒫S\(x\),y<t\)\]\.\\displaystyle\\qquad\-\\log\\pi\_\{\\theta\}\\\!\\left\(y\_\{t\}\\mid\\mathcal\{P\}\_\{S\}\(x\),y\_\{<t\}\\right\)\\Big\]\.

### 3\.4Overall Framework

Figure[2](https://arxiv.org/html/2607.29320#S2.F2)\(c\) summarizes the complete framework\. For each offline input, the student samples a one\-step response and is assigned to one of the three allocation cases in Section[3\.2](https://arxiv.org/html/2607.29320#S3.SS2)\. The corresponding token weightswtw\_\{t\}determine which parts of the routed OPD signal are retained, amplified, or masked\. Meanwhile, the routed domain\-specific teacher scores the same student\-sampled tokens using the type\-hinted prompt in Section[3\.3](https://arxiv.org/html/2607.29320#S3.SS3)\. Combining the student\-side weights with the type\-hinted teacher advantage gives the final objective:

ℒMaga​\(θ\)\\displaystyle\\mathcal\{L\}\_\{\\textsc\{Maga\}\}\(\\theta\)=−𝔼d,x,y\[\\displaystyle=\-\\mathbb\{E\}\_\{d,x,y\}\\Bigg\[\(4\)1\|y\|∑t=1\|y\|wtA^thintlogπθ\(yt∣x,y<t\)\]\.\\displaystyle\\quad\\frac\{1\}\{\|y\|\}\\sum\_\{t=1\}^\{\|y\|\}w\_\{t\}\\widehat\{A\}\_\{t\}^\{\\mathrm\{hint\}\}\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid x,y\_\{<t\}\)\\Bigg\]\.
During training, all domain\-specific teachers remain frozen, and only the student is updated\. When both student\-side allocation and teacher\-side hinting are disabled, the objective reduces to ordinary routed OPD\. Detailed training algorithm is provided in Appendix[A\.3](https://arxiv.org/html/2607.29320#A1.SS3)\.

Table 1:Main comparison on MobileWorld, OSWorld, and WebVoyager across two model scales\. For each domain,Δ​T\\Delta Tis the method SR minus its domain\-specific teacher SR\. SRs are derived from integer success counts over 117, 369, and 140 tasks, respectively\. TNS denotes the Teacher\-Normalized Score defined in Eq\.[5](https://arxiv.org/html/2607.29320#S4.E5)\. Among unified methods within each model scale, the best results are inboldand the second\-best results areunderlined\. Uparrow indicates that a higher value is better

## 4Experiments

In this section, we first describe the experiment implementation details, followed by presenting the main results of our method compared to various baselines and an ablation study\. Then we provide deeper analysis of the trained student’s performance guided by six key questions, revealing fine\-grained behavior beyond aggregate task success\.

### 4\.1Implementation Details

##### Models, Benchmarks, and Baselines\.

In our experiments, we use Qwen3\-VL\-2B and Qwen3\-VL\-8B\(Yang et al\.[2025](https://arxiv.org/html/2607.29320#bib.bib33)\)\. For each model, we first mid\-train the backbone on mixed data from all three domains to obtain a general student\. This checkpoint initializes every trainable student in our experiments\. We then apply SFT separately on each domain to obtain the three frozen domain\-specific teachers\. We test our method on the GUI tasks of MobileWorld\(Kong et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib8)\)and full set of OSWorld\(Xie et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib28)\)\. We also evaluate a subset of WebVoyager\(He et al\.[2024](https://arxiv.org/html/2607.29320#bib.bib5)\), which contains 140 sample tasks\. More information about this subset can be found in Appendix[A\.1](https://arxiv.org/html/2607.29320#A1.SS1.SSS0.Px3)\. The main setting consolidates all three teachers in one run\. The main comparison includes the domain\-specific teachers, SFT, GRPO, Weight Soup\(Wortsman et al\.[2022](https://arxiv.org/html/2607.29320#bib.bib23)\), TIES\(Yadav et al\.[2023](https://arxiv.org/html/2607.29320#bib.bib32)\), and UI\-MOPD\(Lian et al\.[2026](https://arxiv.org/html/2607.29320#bib.bib13)\)\. In the ablation study, we remove the whole student side and its seperate rule\. Besides, we also remove teacher side to demonstrate its effectiveness\. And removing both student and teacher sides ofMagarecovers ordinary per\-sample routed OPD\(Lu and Lab[2025](https://arxiv.org/html/2607.29320#bib.bib16)\)\.

##### Training Details\.

Our training set contains 343k examples, including 93k from mobile, 50k from desktop, and 200k from the web domain\. Notably, the training data for GRPO, UI\-MOPD, andMagais constructed from the same SFT dataset by using the final step’s action as the ground\-truth label\. All runs are conducted on 32 H20 GPUs and optimize the trainable LLM parameters with AdamW, using 1e\-5 as learning rate, batch size 128\. The vision parameters remain frozen\. Each prompt produces 4 rollouts\. After each rollout, the rule\-based reward and parsed action spans determine the student\-side allocation, while the frozen domain\-specific teacher scores the sampled tokens with a training\-only hint\. During inference, we use vLLM\(Kwon et al\.[2023](https://arxiv.org/html/2607.29320#bib.bib9)\)as rollout engine\. Further details are provided in Appendix[A](https://arxiv.org/html/2607.29320#A1)\.

##### Evaluation Protocol\.

We evaluate fixed samples from each domain and use Success Rate \(SR\) as the primary benchmark metric\. Throughout the paper, percentage differences denote absolute differences between percentage\-valued metrics rather than relative changes\. Moreover, to measure how much of each domain\-specific teacher’s capability is retained, we first normalize a model merging method’s SR by the corresponding teacher SR in each domain and then average these ratios\. We define the resulting Teacher\-Normalized Score \(TNS\) for methodmmas

TNS⁡\(m\)=100\|𝒟\|​∑d∈𝒟sm,dsTd,d,\\operatorname\{TNS\}\(m\)=\\frac\{100\}\{\|\\mathcal\{D\}\|\}\\sum\_\{d\\in\\mathcal\{D\}\}\\frac\{s\_\{m,d\}\}\{s\_\{T\_\{d\},d\}\},\(5\)wheresm,ds\_\{m,d\}andsTd,ds\_\{T\_\{d\},d\}are the SRs of methodmmand the domain\-specific teacher on domaindd, respectively\. The higher the TNS is, the more capability of the domain\-specific teachers is retained by the student\. A TNS of100%100\\%matches the domain\-specific teachers on average\. Parameter\-class breakdowns, parsing rules, and statistical tests are provided in Appendix[B](https://arxiv.org/html/2607.29320#A2)\.

### 4\.2Main Results

##### Magaoutperforms the other baselines at both model scales\.

At 8B,Magaexceeds UI\-MOPD on all three domains, improving mean SR by2\.0%2\.0\\%and TNS by4\.4%4\.4\\%\. It also leads Weight Soup and TIES on both metrics\. At 2B, although Weight Soup scores2\.1%2\.1\\%higher on WebVoyager,Magaachieves the highest mean SR and TNS\.

##### Magais comparable to the teachers on average at 8B\.

Across all domains,Magaobtains a TNS of99\.9%99\.9\\%\. Its point estimate is3\.4%3\.4\\%below the MobileWorld teacher but2\.4%2\.4\\%and2\.1%2\.1\\%above the OSWorld and WebVoyager teachers, respectively\. Its mean SR is0\.4%0\.4\\%above the teacher mean\.

Table 2:Ablation study ofMagaon Qwen3\-VL\-8B\. We report SR on MobileWorld, OSWorld, and WebVoyager\.
##### Magareduces the imbalance of weight merging\.

At 8B scale, Weight Soup exceeds the WebVoyager teacher by0\.7%0\.7\\%but falls8\.5%8\.5\\%below the MobileWorld teacher\.Magaimproves these margins to\+2\.1%\+2\.1\\%and−3\.4%\-3\.4\\%, respectively, demonstrating more balanced cross\-domain retention\.

### 4\.3Ablation Study

In Table[2](https://arxiv.org/html/2607.29320#S4.T2), student\-side removal lowers SR by3\.43\.4,3\.83\.8, and1\.41\.4points on MobileWorld, OSWorld, and WebVoyager \. Teacher\-side hint removal lowers MobileWorld and WebVoyager by2\.62\.6and0\.70\.7points, while OSWorld keeps unchanged\. Moreover, ablation results on separate student\-side rules also demonstrate their effectiveness\.

Table 3:Action outcomes of the initial andMaga\-trained students, evaluated on 300 held\-out examples per domain\. Correct denotes a correct action type and all required parameters\. Type×\\timesdenotes responses without a correct action type\. Param\.×\\timesdenotes a correct action type with at least one incorrect or missing required parameter\.![Refer to caption](https://arxiv.org/html/2607.29320v1/x3.png)Figure 3:Based on the trained student, we use 900 random samples \(300 per domain\) to generate response\. \(a\) Each action falls into one of four mutually exclusive groups: matching its corresponding domain\-specific teacher, matching only other domain\-specific teacher, matching no teacher but is correct, or matching no teacher and is incorrect\. Mean values are averaged across domains\. \(b\) The heatmap compares how strongly each response matches its corresponding domain\-specific teacher in the reasoning tokens action type tokens and action parameter tokens\. Darker cells indicate a higher similarity to the teacher\. All reported values are relative to the reasoning tokens, which serve as the zero\-baseline for each domain\.
### 4\.4Do Student\-Side Rules Work as Designed?

##### All three student\-side cases occur, and fully correct actions increase by 21\.0%\.

In Table[3](https://arxiv.org/html/2607.29320#S4.T3), we compare the initial student with the trained student on the same 900 held\-out test examples, with 300 randomly sampled examples from each domain\. These examples are disjoint from the training data\.

Before training,42\.2%42\.2\\%of actions are fully correct,30\.9%30\.9\\%have a wrong action type and26\.9%26\.9\\%have the correct type but an incorrect parameter\. This indicates that all three student\-side allocation cases occur on the held\-out test set, with a majority of errors coming from the action type\.

After training, the percentage of fully correct actions reaches63\.2%63\.2\\%\(\+21\.0%\+21\.0\\%\), while parameter errors fall to22\.2%22\.2\\%\(−4\.7%\-4\.7\\%\) and type errors fall to14\.6%14\.6\\%\(−16\.3%\-16\.3\\%\)\. The increase is consistent across domains, with gains of\+22\.3%\+22\.3\\%on MobileWorld,\+23\.0%\+23\.0\\%on OSWorld, and\+17\.7%\+17\.7\\%onWebVoyager, which indicates that the student’s performance improves across all domains\. Parameter\-class results and task\-level transitions from the initial to the trained student are provided in Appendix[D](https://arxiv.org/html/2607.29320#A4)\.

### 4\.5Where Does Student’s Behavior Come From?

##### The trained student primarily matches the domain\-specific teacher and also produces a small set of successful novel actions\.

Based on 900 randomly sampled offline single\-step tasks \(300 per domain\), we prompt the student and teachers to generate the corresponding answer action, and evaluate whether their action semantics align closely\. Complete evaluation rules are provided in Appendix[A\.5](https://arxiv.org/html/2607.29320#A1.SS5.SSS0.Px2)\.

Figure[3](https://arxiv.org/html/2607.29320#S4.F3)\(a\) shows that 61\.5% of actions match the domain\-specific teacher\. In comparison, 14\.8% match only other domain\-specific teachers\. Thus, the student more often reproduces behavior consistent with its routed domain\-specific teacher than behavior found only in teachers from other domain teachers\.

Most student actions that do not align with any teacher are incorrect, accounting for20\.6%20\.6\\%of all evaluated tasks\. However,3\.0%3\.0\\%of these unmatched actions are actually correct\. These successful yet unaligned behaviors demonstrate that the student is capable of generating valid, novel actions\.

![Refer to caption](https://arxiv.org/html/2607.29320v1/x4.png)Figure 4:For initially incorrect actions, correcting solely the action type allows the model to automatically recover most of the associated parameters\. The student model regenerates all parameters from scratch, without access to the ground\-truth reference parameters\. Each row represents a single response\. The left section groups responses by the ground truth action type, while the right section indicates whether the regenerated action is correct\.

### 4\.6On Which Tokens Do Student and Teacher Agree Most?

Action tokens generated by our trained student show stronger alignment with the corresponding domain\-specific teacher than reasoning tokens\.Figure[3](https://arxiv.org/html/2607.29320#S4.F3)\(b\) compares the likelihood assigned by teachers to reasoning and action tokens generated by our trained student\. The detailed calculation and statistical results are provided in Appendix[A\.5](https://arxiv.org/html/2607.29320#A1.SS5.SSS0.Px3)\.

Compared with reasoning tokens, action tokens show an average likelihood gain of\+0\.072\+0\.072\. Specifically, the type and parameter tokens also show positive average gains of\+0\.068\+0\.068and\+0\.073\+0\.073against reasoning tokens\. This observation supports our action\-aware approach to treat the action tokens as a distinct supervision target rather than weighting all response tokens uniformly\.

### 4\.7Can Action Failures Be Recovered from the Type Alone?

##### Correcting only the action type recovers a substantial percentage of wrong actions\.

From 900 sampled tasks across three domains, we isolate136136responses with incorrect action types\. For these, we replace the predicted action type with the ground truth and prompt the frozen student to regenerate parameters using only its original context and reasoning \(details in Appendix[A\.5](https://arxiv.org/html/2607.29320#A1.SS5.SSS0.Px5)\)\.

As shown in Figure[4](https://arxiv.org/html/2607.29320#S4.F4), correcting only the action type recovers68\.4%68\.4\\%of the responses whose action type is wrong, while the remaining responses still fail due to wrong parameters\. Specifically,5757out of8787actions requiring coordinate parameters are successfully recovered\. This indicates that many wrong actions do not stem from a lack of capability in parameter generation\. Instead, once the action type is corrected, the student model can often generate these parameters correctly\. This finding also supportsMaga’s stronger supervision of the action type\.

### 4\.8CanMagaCorrect Errors on Samples with High\-Disagreement?

##### Magacorrects weight\-merge errors on both Click and Swipe cases\.

To evaluate Weight Soup andMaga, we construct a case study of 66 held\-out MobileWorld tasks \(29Click, 37Swipe\)\. We filtered for spatial actions where all three teachers output valid coordinates with a maximum pairwise distance\>0\.07\>0\.07\.

Table[4](https://arxiv.org/html/2607.29320#S4.T4)reports the corresponding change in accuracy\. Weight Soup fails on 19 samples\. Among them,Magacorrects 6 of the 13 Click errors and 4 of the 6 Swipe errors\. One Click sample changes in the opposite direction, from a correct prediction to an incorrectMagaprediction\. Overall,Magacorrects 10 Weight Soup errors while introducing one new error, increasing accuracy from47/6647/66\(71\.2%71\.2\\%\) to56/6656/66\(84\.8%84\.8\\%\), a gain of13\.6%13\.6\\%\.

### 4\.9Why Does the Student Not Consistently Surpass Its Teachers?

##### Reasoning tokens dominate the response, thereby diluting the action supervision\.

As shown in Table[5](https://arxiv.org/html/2607.29320#S4.T5), we segment each response into reasoning and action spans\. The action accounts for only3\.9%3\.9\\%–7\.1%7\.1\\%of response tokens, making it difficult for one student to exceed every specialized teacher\.

Table 4:MobileWorld case study on high\-disagreement samples\. Model columns report correct actions over the subset size, with accuracy in parentheses\. Change isMagaaccuracy minus Weight Soup accuracy\.Table 5:Response\-token composition and average interaction steps for the 8B student\. The step gap is the failed\-trajectory mean minus the successful\-trajectory mean\.
##### Training on single\-step data limits multi\-step generalization\.

Table[5](https://arxiv.org/html/2607.29320#S4.T5)shows that failed trajectories are longer on average than successful ones, suggesting that long sequences are a bottleneck\. We trainMagausing only single\-step supervision, but real\-world GUI tasks require multi\-step execution, misaligning the multi\-step setting\.

## 5Conclusion

This work addresses how to consolidate multiple domain\-specific GUI teachers into a single agent while retaining their specialized capabilities\. Our analysis shows that weight merging degrades performance, particularly on high\-disagreement spatial samples, while ordinary routed OPD provides only limited training signal to the short structured action\. Moreover, the short action span carries domain\-specific behavior, which significantly affects the environment state\. Therefore, we proposeMagato address this mismatch with structured action signal re\-allocation and a training\-only teacher hint, directing additional supervision according to action correctness and structure\. Across MobileWorld, OSWorld, and WebVoyager at two model scales,Magaachieves the highest mean success rate among unified methods within three domains and best preserves the capabilities of the domain\-specific teachers\. Specifically, at the 8B scale, it improves mean SR by2\.0%2\.0\\%over the strongest baseline and achieves almost the same average performance with teachers, suggesting that robust multi\-platform consolidation benefits from explicit supervision of structured actions\.

## References

- Cheng et al\. \(2026\)Cheng, K\.; Li, Z\.; Ma, Z\.; Chen, N\.; Cao, J\.; Sun, Q\.; Ding, Z\.; Xu, F\.; Yan, H\.; Chen, J\.; et al\. 2026\.OpenMobile: Building open mobile agents with task and trajectory synthesis\.*arXiv preprint arXiv:2604\.15093*\.
- Cheng et al\. \(2024\)Cheng, K\.; Sun, Q\.; Chu, Y\.; Xu, F\.; YanTao, L\.; Zhang, J\.; and Wu, Z\. 2024\.Seeclick: Harnessing gui grounding for advanced visual gui agents\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, 9313–9332\.
- Gong et al\. \(2026\)Gong, Y\.; Cai, Z\.; Dai, S\.; Zhou, Y\.; Gu, Z\.; Meng, C\.; and Shen, S\. 2026\.VenusBench\-Mobile: A Challenging and User\-Centric Benchmark for Mobile GUI Agents with Capability Diagnostics\.In*Forty\-third International Conference on Machine Learning*\.
- Gu et al\. \(2025\)Gu, Z\.; Zeng, Z\.; Xu, Z\.; Zhou, X\.; Shen, S\.; Liu, Y\.; Zhou, B\.; Meng, C\.; Xia, T\.; Chen, W\.; et al\. 2025\.Ui\-venus technical report: Building high\-performance ui agents with rft\.*arXiv preprint arXiv:2508\.10833*\.
- He et al\. \(2024\)He, H\.; Yao, W\.; Ma, K\.; Yu, W\.; Dai, Y\.; Zhang, H\.; Lan, Z\.; and Yu, D\. 2024\.Webvoyager: Building an end\-to\-end web agent with large multimodal models\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, 6864–6890\.
- Hu et al\. \(2026\)Hu, J\.; Liu, J\.; Lai, J\.; Hu, J\.; Sheng, Y\.; Chen, S\.; Li, J\.; Du, D\.; and Guo, S\. 2026\.GUI Agents with Reinforcement Learning: Toward Digital Inhabitants\.*arXiv preprint arXiv:2604\.27955*\.
- Jia et al\. \(2025\)Jia, C\.; Luo, M\.; Dang, Z\.; Sun, Q\.; Xu, F\.; Hu, J\.; Xie, T\.; and Wu, Z\. 2025\.Agentstore: Scalable integration of heterogeneous agents as specialized generalist computer assistant\.In*Findings of the Association for Computational Linguistics: ACL 2025*, 8908–8934\.
- Kong et al\. \(2026\)Kong, Q\.; Zhang, X\.; Yang, Z\.; Gao, N\.; Liu, C\.; Tong, P\.; Cai, C\.; Zhou, H\.; Zhang, J\.; Chen, L\.; et al\. 2026\.Mobileworld: Benchmarking autonomous mobile agents in agent\-user interactive and mcp\-augmented environments\.In*Proceedings of the 64th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, 6142–6167\.
- Kwon et al\. \(2023\)Kwon, W\.; Li, Z\.; Zhuang, S\.; Sheng, Y\.; Zheng, L\.; Yu, C\. H\.; Gonzalez, J\. E\.; Zhang, H\.; and Stoica, I\. 2023\.Efficient Memory Management for Large Language Model Serving with PagedAttention\.In*Proceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles*\.
- Li et al\. \(2026a\)Li, J\.; Deng, K\.; Wang, Y\.; Huang, J\.; Shi, Y\.; Tan, Q\.; Lu, J\.; and Liu, N\. 2026a\.Online Skill Learning for Web Agents via State\-Grounded Dynamic Retrieval\.*arXiv preprint arXiv:2606\.04391*\.
- Li et al\. \(2026b\)Li, Y\.; Zuo, Y\.; He, B\.; Zhang, J\.; Xiao, C\.; Qian, C\.; Yu, T\.; Gao, H\.\-a\.; Yang, W\.; Liu, Z\.; et al\. 2026b\.Rethinking on\-policy distillation of large language models: Phenomenology, mechanism, and recipe\.*arXiv preprint arXiv:2604\.13016*\.
- Li et al\. \(2025\)Li, Z\.; Wu, X\.; Du, H\.; Liu, F\.; Nghiem, H\.; and Shi, G\. 2025\.A survey of state of the art large vision language models: Benchmark evaluations and challenges\.In*Proceedings of the Computer Vision and Pattern Recognition Conference*, 1587–1606\.
- Lian et al\. \(2026\)Lian, N\.; Chen, A\.; Yu, Z\.; Duan, C\.; Liu, F\.; Liu, H\.; Fu, P\.; Luan, J\.; Wang, Y\.; Xia, S\.\-T\.; and Wang, J\. 2026\.UI\-MOPD: Multi\-Platform On\-Policy Distillation for Continual GUI Agent Learning\.arXiv:2607\.04425\.
- Liang et al\. \(2026\)Liang, C\. X\.; Tian, P\.; Yin, C\. H\.; Yua, Y\.; Wei, A\.\-H\.; Li, M\.; Song, X\.; Wang, T\.; Bi, Z\.; Liu, M\.; et al\. 2026\.A comprehensive survey and guide to multimodal large language models in vision–language tasks\.*Computation*, 14\(6\): 125\.
- Liu et al\. \(2025\)Liu, Z\.; Xie, J\.; Ding, Z\.; Li, Z\.; Yang, B\.; Wu, Z\.; Wang, X\.; Sun, Q\.; Liu, S\.; Wang, W\.; et al\. 2025\.Scalecua: Scaling open\-source computer use agents with cross\-platform data\.*arXiv preprint arXiv:2509\.15221*\.
- Lu and Lab \(2025\)Lu, K\.; and Lab, T\. M\. 2025\.On\-Policy Distillation\.*Thinking Machines Lab: Connectionism*\.Https://thinkingmachines\.ai/blog/on\-policy\-distillation\.
- Luong et al\. \(2024\)Luong, T\. Q\.; Zhang, X\.; Jie, Z\.; Sun, P\.; Jin, X\.; and Li, H\. 2024\.Reft: Reasoning with reinforced fine\-tuning\.*arXiv preprint arXiv:2401\.08967*\.
- Shao et al\. \(2024\)Shao, Z\.; Wang, P\.; Zhu, Q\.; Xu, R\.; Song, J\.; Bi, X\.; Zhang, H\.; Zhang, M\.; Li, Y\.; Wu, Y\.; et al\. 2024\.Deepseekmath: Pushing the limits of mathematical reasoning in open language models, 2024\.*URL https://arxiv\. org/abs/2402\.03300*, 2\(3\): 5\.
- Sun et al\. \(2025\)Sun, Q\.; Cheng, K\.; Ding, Z\.; Jin, C\.; Wang, Y\.; Xu, F\.; Wu, Z\.; Jia, C\.; Chen, L\.; Liu, Z\.; et al\. 2025\.Os\-genesis: Automating gui agent trajectory construction via reverse task synthesis\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, 5555–5579\.
- Tang et al\. \(2026a\)Tang, F\.; Gu, Z\.; Lu, Z\.; Liu, X\.; Shen, S\.; Meng, C\.; Wang, W\.; Zhang, W\.; Shen, Y\.; Lu, W\.; et al\. 2026a\.GUI\-G2: Gaussian Reward Modeling for GUI Grounding\.In*Proceedings of the AAAI Conference on Artificial Intelligence*, volume 40, 33214–33222\.
- Tang et al\. \(2026b\)Tang, F\.; Gu, Z\.; Lu, Z\.; Zhang, S\.; Zeng, Z\.; Shen, S\.; Meng, C\.; Yan, Y\.; Zhang, W\.; Shen, Y\.; et al\. 2026b\.GUI\-SAGE: Enhancing GUI Automation with Self\-Explanatory Learning\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 13007–13016\.
- Team et al\. \(2026\)Team, V\.; Gao, C\.; Gu, Z\.; Liu, Y\.; Qiu, X\.; Shen, S\.; Wen, Y\.; Xia, T\.; Xu, Z\.; Zeng, Z\.; et al\. 2026\.Ui\-venus\-1\.5 technical report\.*arXiv preprint arXiv:2602\.09082*\.
- Wortsman et al\. \(2022\)Wortsman, M\.; Ilharco, G\.; Gadre, S\. Y\.; Roelofs, R\.; Gontijo\-Lopes, R\.; Morcos, A\. S\.; Namkoong, H\.; Farhadi, A\.; Carmon, Y\.; Kornblith, S\.; et al\. 2022\.Model soups: averaging weights of multiple fine\-tuned models improves accuracy without increasing inference time\.In*International conference on machine learning*, 23965–23998\. PMLR\.
- Wu et al\. \(2026a\)Wu, J\.; Yang, S\.; Lu, Z\.; Zhang, F\.; Shen, Y\.; Feng, L\.; Luo, H\.; Lian, Z\.; Zhang, S\.; Wen, Z\.; and Tao, J\. 2026a\.SEED: Self\-Evolving On\-Policy Distillation for Agentic Reinforcement Learning\.arXiv:2607\.14777\.
- Wu et al\. \(2026b\)Wu, J\.; Yang, S\.; Lu, Z\.; Zhang, F\.; Shen, Y\.; Feng, L\.; Luo, H\.; Lian, Z\.; Zhang, S\.; Wen, Z\.; et al\. 2026b\.SEED: Self\-Evolving On\-Policy Distillation for Agentic Reinforcement Learning\.*arXiv preprint arXiv:2607\.14777*\.
- Wu et al\. \(2025\)Wu, Z\.; Wu, Z\.; Xu, F\.; Wang, Y\.; Sun, Q\.; Jia, C\.; Cheng, K\.; Ding, Z\.; Chen, L\.; Liang, P\. P\.; et al\. 2025\.OS\-ATLAS: Foundation action model for generalist GUI agents\.In*International Conference on Learning Representations*, volume 2025, 5090–5108\.
- Xiao et al\. \(2026\)Xiao, B\.; Xia, B\.; Yang, B\.; Gao, B\.; Shen, B\.; Zhang, C\.; He, C\.; Lou, C\.; Luo, F\.; Wang, G\.; et al\. 2026\.Mimo\-v2\-flash technical report\.*arXiv preprint arXiv:2601\.02780*\.
- Xie et al\. \(2024\)Xie, T\.; Zhang, D\.; Chen, J\.; Li, X\.; Zhao, S\.; Cao, R\.; Hua, T\. J\.; Cheng, Z\.; Shin, D\.; Lei, F\.; et al\. 2024\.Osworld: Benchmarking multimodal agents for open\-ended tasks in real computer environments\.*Advances in Neural Information Processing Systems*, 37: 52040–52094\.
- Xu et al\. \(2026a\)Xu, A\.; Lin, B\.; Xue, B\.; Wang, B\.; Xu, B\.; Wu, B\.; Zhang, B\.; Lin, C\.; Dong, C\.; Ling, C\.; et al\. 2026a\.Deepseek\-v4: Towards highly efficient million\-token context intelligence\.*arXiv preprint arXiv:2606\.19348*\.
- Xu et al\. \(2026b\)Xu, H\.; Zhang, X\.; Liu, H\.; Wang, J\.; Zhu, Z\.; Zhou, S\.; Hu, X\.; Gao, F\.; Cao, J\.; Wang, Z\.; et al\. 2026b\.Mobile\-agent\-v3\. 5: Multi\-platform fundamental gui agents\.*arXiv preprint arXiv:2602\.16855*\.
- Xue et al\. \(2026\)Xue, T\.; Peng, C\.; Huang, M\.; Guo, L\.; Han, T\.; Wang, H\.; Wang, J\.; Zhang, X\.; Yang, X\.; Zhao, D\.; et al\. 2026\.Evocua: Evolving computer use agents via learning from scalable synthetic experience\.*arXiv preprint arXiv:2601\.15876*\.
- Yadav et al\. \(2023\)Yadav, P\.; Tam, D\.; Choshen, L\.; Raffel, C\. A\.; and Bansal, M\. 2023\.Ties\-merging: Resolving interference when merging models\.*Advances in neural information processing systems*, 36: 7093–7115\.
- Yang et al\. \(2025\)Yang, A\.; Li, A\.; Yang, B\.; Zhang, B\.; Hui, B\.; Zheng, B\.; Yu, B\.; Gao, C\.; Huang, C\.; Lv, C\.; et al\. 2025\.Qwen3 technical report\.*arXiv preprint arXiv:2505\.09388*\.
- Yang et al\. \(2026a\)Yang, B\.; Jin, K\.; Wu, Z\.; Liu, Z\.; Sun, Q\.; Li, Z\.; Xie, J\.; Liu, Z\.; Xu, F\.; Cheng, K\.; et al\. 2026a\.Os\-symphony: A holistic framework for robust and generalist computer\-using agent\.*arXiv preprint arXiv:2601\.07779*\.
- Yang et al\. \(2026b\)Yang, S\.; Wu, J\.; Lu, Z\.; Shen, Y\.; Zhang, F\.; Feng, L\.; Zhang, S\.; Luo, H\.; Lian, Z\.; Wen, Z\.; and Tao, J\. 2026b\.OPID: On\-Policy Skill Distillation for Agentic Reinforcement Learning\.arXiv:2606\.26790\.
- Yang et al\. \(2026c\)Yang, S\.; Wu, J\.; Lu, Z\.; Shen, Y\.; Zhang, F\.; Feng, L\.; Zhang, S\.; Luo, H\.; Lian, Z\.; Wen, Z\.; et al\. 2026c\.OPID: On\-Policy Skill Distillation for Agentic Reinforcement Learning\.*arXiv preprint arXiv:2606\.26790*\.
- Yao et al\. \(2022\)Yao, S\.; Chen, H\.; Yang, J\.; and Narasimhan, K\. 2022\.Webshop: Towards scalable real\-world web interaction with grounded language agents\.*Advances in Neural Information Processing Systems*, 35: 20744–20757\.
- Zhang et al\. \(2026a\)Zhang, J\.; Ji, Y\.; Ren, F\.; Li, Y\.; Zeng, B\.; Chen, Z\.; Chen, K\.; Shou, L\.; Chen, G\.; and Li, H\. 2026a\.Efficient inference for large vision\-language models: Bottlenecks, techniques, and prospects\.In*Findings of the Association for Computational Linguistics: ACL 2026*, 21036–21066\.
- Zhang et al\. \(2026b\)Zhang, X\.; Dong, Y\.; Zhang, L\.; Jia, C\.; Dang, Z\.; Fernando, B\.; Liu, J\.; and Shou, M\. Z\. 2026b\.Cofft: Chain of foresight\-focus thought for visual language models\.*Advances in Neural Information Processing Systems*, 38: 106182–106201\.
- Zhang et al\. \(2026c\)Zhang, X\.; Dong, Y\.; Zhang, L\.; Jia, C\.; Dang, Z\.; Yao, Y\.; Wu, Y\.; Fernando, B\.; and Liu, J\. 2026c\.Beyond Layer\-Wise Merging: Chain\-of\-Merging for Vision\-Language Models\.In*Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition*, 24279–24289\.
- Zhang et al\. \(2026d\)Zhang, Y\.; Tang, S\.; Li, Z\.; Han, Z\.; and Tresp, V\. 2026d\.WebArbiter: A Principle\-Guided Reasoning Process Reward Model for Web Agents\.*arXiv preprint arXiv:2601\.21872*\.
- Zheng et al\. \(2024\)Zheng, B\.; Gou, B\.; Kil, J\.; Sun, H\.; and Su, Y\. 2024\.Gpt\-4v \(ision\) is a generalist web agent, if grounded\.*arXiv preprint arXiv:2401\.01614*\.
- Zhou et al\. \(2025a\)Zhou, B\.; Huang, Z\.; Guo, Y\.; Gu, Z\.; Xia, T\.; Luo, Z\.; Tang, F\.; Kong, D\.; Shang, Y\.; Ou, S\.; et al\. 2025a\.VenusBench\-GD: A Comprehensive Multi\-Platform GUI Benchmark for Diverse Grounding Tasks\.*arXiv preprint arXiv:2512\.16501*\.
- Zhou et al\. \(2025b\)Zhou, H\.; Zhang, X\.; Tong, P\.; Zhang, J\.; Chen, L\.; Kong, Q\.; Cai, C\.; Liu, C\.; Wang, Y\.; Zhou, J\.; et al\. 2025b\.MAI\-UI Technical Report: Real\-World Centric Foundation GUI Agents\.*arXiv preprint arXiv:2512\.22047*\.
- Zhu et al\. \(2024\)Zhu, D\.; Shen, X\.; Li, X\.; Elhoseiny, M\.; et al\. 2024\.Minigpt\-4: Enhancing vision\-language understanding with advanced large language models\.In*International Conference on Learning Representations*, volume 2024, 18378–18394\.

## Appendix AExperimental Implementation Details

### A\.1Checkpoints and Data

##### Checkpoint construction\.

For each model scale, we mid\-train Qwen3\-VL on the mixed mobile, desktop, and web data to obtain a shared general\-student checkpoint\. Every trainable student in the main comparison, ablations, and analyses is initialized from this checkpoint\. We independently apply domain\-specific SFT to three copies of the same checkpoint to obtain the MobileWorld, OSWorld, and WebVoyager teachers\. These domain\-specific teachers remain frozen during distillation\.

##### Training data\.

The data are subject to privacy requirements and non\-disclosure agreements and therefore cannot be released\. The collection differs from available public datasets in scale, domain composition, and production conditions\. Our training corpus contains 343k proprietary industrial examples\. 200k from the web domain, 93k from the MobileWorld domain, and 50k from the OSWorld domain\.

##### WebVoyager subset\.

The fixed evaluation subset contains 140 tasks, with ten tasks from each of 14 websites\. The task IDs are listed below\.

Allrecipes\.`Allrecipes\-\-43`,`Allrecipes\-\-6`,`Allrecipes\-\-11`,`Allrecipes\-\-17`,`Allrecipes\-\-34`,`Allrecipes\-\-2`,`Allrecipes\-\-14`,`Allrecipes\-\-36`,`Allrecipes\-\-41`,`Allrecipes\-\-35`\.

Amazon\.`Amazon\-\-32`,`Amazon\-\-35`,`Amazon\-\-17`,`Amazon\-\-33`,`Amazon\-\-8`,`Amazon\-\-28`,`Amazon\-\-12`,`Amazon\-\-10`,`Amazon\-\-14`,`Amazon\-\-37`\.

Apple\.`Apple\-\-13`,`Apple\-\-21`,`Apple\-\-33`,`Apple\-\-10`,`Apple\-\-24`,`Apple\-\-38`,`Apple\-\-19`,`Apple\-\-18`,`Apple\-\-26`,`Apple\-\-34`\.

ArXiv\.`ArXiv\-\-7`,`ArXiv\-\-30`,`ArXiv\-\-37`,`ArXiv\-\-2`,`ArXiv\-\-14`,`ArXiv\-\-20`,`ArXiv\-\-35`,`ArXiv\-\-33`,`ArXiv\-\-17`,`ArXiv\-\-23`\.

BBC News\.`BBC News\-\-40`,`BBC News\-\-3`,`BBC News\-\-6`,`BBC News\-\-15`,`BBC News\-\-35`,`BBC News\-\-26`,`BBC News\-\-19`,`BBC News\-\-24`,`BBC News\-\-36`,`BBC News\-\-17`\.

Booking\.`Booking\-\-15`,`Booking\-\-0`,`Booking\-\-29`,`Booking\-\-8`,`Booking\-\-43`,`Booking\-\-35`,`Booking\-\-19`,`Booking\-\-18`,`Booking\-\-22`,`Booking\-\-33`\.

Coursera\.`Coursera\-\-21`,`Coursera\-\-29`,`Coursera\-\-31`,`Coursera\-\-14`,`Coursera\-\-16`,`Coursera\-\-37`,`Coursera\-\-40`,`Coursera\-\-41`,`Coursera\-\-23`,`Coursera\-\-18`\.

ESPN\.`ESPN\-\-34`,`ESPN\-\-21`,`ESPN\-\-38`,`ESPN\-\-12`,`ESPN\-\-14`,`ESPN\-\-41`,`ESPN\-\-26`,`ESPN\-\-8`,`ESPN\-\-37`,`ESPN\-\-1`\.

GitHub\.`GitHub\-\-16`,`GitHub\-\-27`,`GitHub\-\-25`,`GitHub\-\-5`,`GitHub\-\-7`,`GitHub\-\-18`,`GitHub\-\-17`,`GitHub\-\-0`,`GitHub\-\-39`,`GitHub\-\-10`\.

Google Flights\.`Google Flights\-\-6`,`Google Flights\-\-23`,`Google Flights\-\-35`,`Google Flights\-\-8`,`Google Flights\-\-15`,`Google Flights\-\-16`,`Google Flights\-\-3`,`Google Flights\-\-20`,`Google Flights\-\-22`,`Google Flights\-\-5`\.

Google Map\.`Google Map\-\-8`,`Google Map\-\-9`,`Google Map\-\-5`,`Google Map\-\-19`,`Google Map\-\-22`,`Google Map\-\-20`,`Google Map\-\-17`,`Google Map\-\-39`,`Google Map\-\-11`,`Google Map\-\-28`\.

Google Search\.`Google Search\-\-3`,`Google Search\-\-24`,`Google Search\-\-21`,`Google Search\-\-35`,`Google Search\-\-27`,`Google Search\-\-20`,`Google Search\-\-41`,`Google Search\-\-9`,`Google Search\-\-38`,`Google Search\-\-5`\.

Huggingface\.`Huggingface\-\-36`,`Huggingface\-\-33`,`Huggingface\-\-17`,`Huggingface\-\-37`,`Huggingface\-\-7`,`Huggingface\-\-4`,`Huggingface\-\-5`,`Huggingface\-\-8`,`Huggingface\-\-29`,`Huggingface\-\-12`\.

Wolfram Alpha\.`Wolfram Alpha\-\-36`,`Wolfram Alpha\-\-5`,`Wolfram Alpha\-\-32`,`Wolfram Alpha\-\-8`,`Wolfram Alpha\-\-1`,`Wolfram Alpha\-\-0`,`Wolfram Alpha\-\-33`,`Wolfram Alpha\-\-17`,`Wolfram Alpha\-\-29`,`Wolfram Alpha\-\-41`\.

### A\.2Training and Baseline Configurations

##### Compute and shared training setup\.

All training runs use 32 H20 GPUs for one epoch\. We optimize the trainable LLM parameters with AdamW using a global batch size of 128, zero weight decay, and a maximum sequence length of 16,384 tokens\. The learning rate increases linearly during the first 3% of training and then remains constant at1×10−51\\times 10^\{\-5\}\. Each prompt produces four student rollouts\. Each training example contains one output action, although its input may include interaction history\. All distillation variants use the same student initialization, routed domain data, and frozen domain\-specific teachers\. The vision tower is bit\-identical across teachers and remains frozen throughout training\. Besides, we separately disable the three rules in the student\-side allocation to evaluate their effectiveness\.

##### Baseline and ablation controls\.

The main comparison uses the complete method, including student\-side allocation and the teacher\-side hint\. The component ablation disables either operation while keeping the remaining setup fixed\. Disabling both recovers ordinary routed OPD exactly\.

##### Amplification coefficient\.

We setβ=1\\beta=1in all experiments, which doubles the weight of the selected action tokens, and keep it fixed across model scales and domains without domain\-specific tuning\.

### A\.3Hint and Allocation Implementation

##### Training algorithm\.

Algorithm[1](https://arxiv.org/html/2607.29320#alg1)summarizes the distributed implementation from launch through actor updates\. The worker countPPand resource placement are read from the training configuration rather than hard\-coded in the method\. The configured update countKKcovers the single training epoch\. The Ray reward actor supplies only the exact\-action acceptance gate; the actor update uses the routed reverse\-KL distillation advantage and does not optimize a task\-reward objective\.

For sampleii, letai∗=\(zi∗,pi∗\)a\_\{i\}^\{\*\}=\(z\_\{i\}^\{\*\},p\_\{i\}^\{\*\}\)be the reference action and letyi=\(yi,1,…,yi,\|yi\|\)y\_\{i\}=\(y\_\{i,1\},\\ldots,y\_\{i,\|y\_\{i\}\|\}\)be the response sampled from the student\. At token positiontt,ℓi,tT\\ell\_\{i,t\}^\{T\}is the log probability assigned to the sampled token by the routed teacher using the type\-hinted prompt, whereasℓi,tS\\ell\_\{i,t\}^\{S\}is the log probability assigned by the student using the original prompt:

ℓi,tT\\displaystyle\\ell\_\{i,t\}^\{T\}=log⁡πTdi​\(yi,t∣𝒫T​\(xi,zi∗\),yi,<t\),\\displaystyle=\\log\\pi\_\{T\_\{d\_\{i\}\}\}\(y\_\{i,t\}\\mid\\mathcal\{P\}\_\{T\}\(x\_\{i\},z\_\{i\}^\{\*\}\),y\_\{i,<t\}\),ℓi,tS\\displaystyle\\ell\_\{i,t\}^\{S\}=log⁡πθ​\(yi,t∣𝒫S​\(xi\),yi,<t\)\.\\displaystyle=\\log\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid\\mathcal\{P\}\_\{S\}\(x\_\{i\}\),y\_\{i,<t\}\)\.The three binary masks identify the reasoning, action\-type, and parameter positions in the sampled response:

mi,tr\\displaystyle m\_\{i,t\}^\{r\}=𝟏​\[t∈ℐreason\],\\displaystyle=\\mathbf\{1\}\[t\\in\\mathcal\{I\}\_\{\\mathrm\{reason\}\}\],mi,tz\\displaystyle m\_\{i,t\}^\{z\}=𝟏​\[t∈ℐtype\],\\displaystyle=\\mathbf\{1\}\[t\\in\\mathcal\{I\}\_\{\\mathrm\{type\}\}\],mi,tp\\displaystyle m\_\{i,t\}^\{p\}=𝟏​\[t∈ℐparam\]\.\\displaystyle=\\mathbf\{1\}\[t\\in\\mathcal\{I\}\_\{\\mathrm\{param\}\}\]\.These disjoint masks partition the response tokens\. Parameter tokens include coordinates, text, URLs, keys, and other values required by the predicted type\-specific schema\. We usez^i=ParseType​\(yi\)\\hat\{z\}\_\{i\}=\\textsc\{ParseType\}\(y\_\{i\}\)for the type parsed from the student response\. Brackets denote a binary indicator, so\[z^i≠Fail∧z^i=zi∗\]\[\\hat\{z\}\_\{i\}\\neq\\textsc\{Fail\}\\land\\hat\{z\}\_\{i\}=z\_\{i\}^\{\*\}\]equals one exactly when parsing succeeds and the predicted type matches the reference type, and equals zero otherwise\.

Algorithm 1MagaTraining Algorithm0:Config

𝒞\\mathcal\{C\}; routed data

𝒟\\mathcal\{D\}; student

πθ\\pi\_\{\\theta\}; teachers

\{Td\}\\\{T\_\{d\}\\\}; coefficient

β\\beta
1:Stage 0: launchRead

PPand

KKfrom

𝒞\\mathcal\{C\}; set

k←0k\\leftarrow 0; useTorchrunto start

PPprocesses; initialize Ray and one runner\.

2:Stage 1: initializeLoad the tokenizer/processor, dataloader, reward actors, and shared GPU resource pool\.

3:for all

p∈\{1,…,P\}p\\in\\\{1,\\ldots,P\\\}in paralleldo

4:Load the sharded actor, vLLM rollout engine, and frozen teachers into one FSDP worker\.

5:endfor

6:Stage 2: train

7:while

k<Kk<Kdo

8:

X←NextBatch​\(𝒟\)X\\leftarrow\\textsc\{NextBatch\}\(\\mathcal\{D\}\);

B←ExpandEachPrompt​\(X,4\)B\\leftarrow\\textsc\{ExpandEachPrompt\}\(X,4\)\.

9:

yi∼πθ\(⋅∣xi\)y\_\{i\}\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\_\{i\}\)for each rollout slot

i∈Bi\\in B\.

10:Asynchronously compute

ci←ExactAccept​\(yi,ai∗\)c\_\{i\}\\leftarrow\\textsc\{ExactAccept\}\(y\_\{i\},a\_\{i\}^\{\*\}\)for each rollout\.

11:for all

i∈Bi\\in Bin paralleldo

12:Route source

did\_\{i\}to

TdiT\_\{d\_\{i\}\}; set

hiT=\(xi,TypeHint​\(zi∗\)\)h\_\{i\}^\{T\}=\(x\_\{i\},\\textsc\{TypeHint\}\(z\_\{i\}^\{\*\}\)\)\.

13:Score

yiy\_\{i\}under

TdiT\_\{d\_\{i\}\}with

hiTh\_\{i\}^\{T\}and under

πθ\\pi\_\{\\theta\}with

xix\_\{i\}to obtain

ℓi,tT\\ell\_\{i,t\}^\{T\}and

ℓi,tS\\ell\_\{i,t\}^\{S\}\.

14:Build

mi,tr,mi,tz,mi,tpm\_\{i,t\}^\{r\},m\_\{i,t\}^\{z\},m\_\{i,t\}^\{p\}from the sampled token IDs\.

15:

z^i←ParseType​\(yi\)\\hat\{z\}\_\{i\}\\leftarrow\\textsc\{ParseType\}\(y\_\{i\}\);

qi←\[z^i≠Fail∧z^i=zi∗\]q\_\{i\}\\leftarrow\[\\hat\{z\}\_\{i\}\\neq\\textsc\{Fail\}\\land\\hat\{z\}\_\{i\}=z\_\{i\}^\{\*\}\]\.

16:

A^i,tH←sg⁡\[ℓi,tT−ℓi,tS\]\\widehat\{A\}\_\{i,t\}^\{H\}\\leftarrow\\operatorname\{sg\}\[\\ell\_\{i,t\}^\{T\}\-\\ell\_\{i,t\}^\{S\}\]\.

17:if

cic\_\{i\}then

18:

wi,t←0w\_\{i,t\}\\leftarrow 0\{fully accepted\}

19:elseif

qiq\_\{i\}then

20:

wi,t←mi,tr\+\(1\+β\)​\(mi,tz\+mi,tp\)w\_\{i,t\}\\leftarrow m\_\{i,t\}^\{r\}\+\(1\+\\beta\)\(m\_\{i,t\}^\{z\}\+m\_\{i,t\}^\{p\}\)\{amplify action\}

21:else

22:

wi,t←mi,tr\+\(1\+β\)​mi,tzw\_\{i,t\}\\leftarrow m\_\{i,t\}^\{r\}\+\(1\+\\beta\)m\_\{i,t\}^\{z\}\{amplify type; mask params\}

23:endif

24:

A^i,tMaga←wi,t​A^i,tH\\widehat\{A\}\_\{i,t\}^\{\\textsc\{Maga\}\}\\leftarrow w\_\{i,t\}\\widehat\{A\}\_\{i,t\}^\{H\}\.

25:endfor

26:

ℒ←−1\|B\|​∑i∈B1\|yi\|​∑tA^i,tMaga​log⁡πθ​\(yi,t∣xi,yi,<t\)\\mathcal\{L\}\\leftarrow\-\\frac\{1\}\{\|B\|\}\\sum\_\{i\\in B\}\\frac\{1\}\{\|y\_\{i\}\|\}\\sum\_\{t\}\\widehat\{A\}\_\{i,t\}^\{\\textsc\{Maga\}\}\\log\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid x\_\{i\},y\_\{i,<t\}\)\.

27:Update

28:

k←k\+1k\\leftarrow k\+1\.

29:endwhile

30:Stage 3: finalizeSave the actor, TensorBoard events, and per\-token distillation logs\.

##### Teacher\-side hint\.

For each training sample, we construct a text hint from only the ground\-truth action using the templateHint: the correct action for this step is a \{HINT\} action\.We append the hint to the selected domain\-specific teacher’s user turn, after the original instruction and before the assistant response\. The teacher then scores exactly the student\-generated response tokens\. Disabling the hint restores the original teacher prompt\.

##### Mask construction\.

We preserve ordinary routed OPD trajectory collection and build the allocation masks after rollout\. Incremental cumulative decoding locates the reasoning, type, parameter, and action spans without re\-encoding the response or invoking another model\. The action\-acceptance component of the existing rollout reward determines whether the sampled action is fully correct\. The selected domain\-specific teacher then scores the student tokens without decoding\. Under an incorrect type, parameter tokens are masked, while only the type span receives additional weight and reasoning retains its ordinary distillation advantage\.

### A\.4Exact Action Evaluator

##### Exact action acceptance\.

The correctness gate reuses the action\-acceptance component of the existing rule\-based training reward\. It is an offline evaluator signal, not task success from live interaction\. A sampled action is fully correct only when the evaluator accepts its type and every parameter required by the reference type\-specific schema\. Partial parameter credit does not pass the gate\. The separate response\- format reward is unchanged by our method but does not determine student\-side allocation\.

Both the predicted and reference actions are parsed into an action type and its type\-specific parameters\. A parse failure or type mismatch fails the action\-acceptance gate\. An unparseable action follows the wrong\-type branch that the implementation retains ordinary reasoning supervision, strengthens an identifiable type token when available, and masks tokens assigned to the parameter span\. Once the type matches, the evaluator applies the action\-specific parameter rule in Table[6](https://arxiv.org/html/2607.29320#A1.T6)\.

Table 6:Correctness rules for each type\-specific parameter\. Partial credit does not pass the binary allocation gate\.

### A\.5Analysis Protocols

##### Allocation\-profile experiment\.

The allocation profile compares the Qwen3\-VL\-2B student checkpoint with its correspondingMaga\-trained checkpoint\. We sample 300 held\-out test examples per domain\. These examples are disjoint from the training data\. Prompts longer than 14k tokens are excluded rather than truncated\. Both checkpoints use the same prompt template and rollout settings, with temperature0\.90\.9, top\-pp1\.01\.0, no top\-kktruncation, and a maximum completion length of 1024 tokens\. All 900 task IDs are matched between checkpoints for the transition analysis\.

##### Teacher\-matching analysis\.

We compare the trained student’s parsed final action with the actions produced by all three frozen domain\-specific teachers on the same offline single\-step task\. The matched set contains 900 tasks, with 300 from each domain\. An action match requires the same action type and accepted type\-specific parameters\. Coordinate\-valued parameters use the0\.070\.07normalized\-distance threshold\. A shared match means that the student matches at least two teachers\. For the main figure, all actions matching the domain\-specific teacher are merged into one category, whether or not they also match another domain\-specific teacher\. Actions that do not match the domain\-specific teacher are divided into matches with other domain\-specific teachers and matches with no teacher\. No\-teacher matches are divided by whether the existing evaluator accepts the student action for the reference step\.

##### Spanwise teacher\-likelihood analysis\.

All three frozen teachers score every response under teacher forcing, without decoding a new response\. LetMd,t\(s\)M\_\{d,t\}^\{\(s\)\}denote the mean length\-normalized log likelihood assigned by teacherttto spanssof responses from domaindd\. Because raw scores contain both response\-domain and teacher\-wide calibration effects, we compute the two\-way\-centered interaction

Rd,t\(s\)=Md,t\(s\)−M¯d,⋅\(s\)−M¯⋅,t\(s\)\+M¯⋅,⋅\(s\)\.R\_\{d,t\}^\{\(s\)\}=M\_\{d,t\}^\{\(s\)\}\-\\bar\{M\}\_\{d,\\cdot\}^\{\(s\)\}\-\\bar\{M\}\_\{\\cdot,t\}^\{\(s\)\}\+\\bar\{M\}\_\{\\cdot,\\cdot\}^\{\(s\)\}\.\(6\)For each domain, the reported value for spanssisEd\(s\)=Rd,d\(s\)−Rd,d\(reasoning\)E\_\{d\}^\{\(s\)\}=R\_\{d,d\}^\{\(s\)\}\-R\_\{d,d\}^\{\(\\mathrm\{reasoning\}\)\}\. Reasoning is therefore zero by construction, and positive values indicate a larger domain–teacher interaction than on reasoning tokens\. The type and parameter columns are diagnostic subspans of the full action and are not additive\. We compute uncertainty with 2,000 domain\-stratified bootstrap resamples over task IDs and macro\-average only after computing the statistic within each domain\.

##### Disagreement\-conditioned diagnostics\.

We first select samples with a reference spatial action and parseable coordinates from all three frozen teachers\. High disagreement means that the maximum pairwise teacher distance exceeds the single\-step acceptance radius \(0\.070\.07\)\.

##### Oracle type intervention\.

We greedily decode900900randomly sampled responses\. We retain responses whose predicted type is wrong and parseable, whose reference action requires parameters, and whose type span can be located reliably\. This gives136136responses:3636from MobileWorld,5252from OSWorld, and4848from WebVoyager\.

For each response, we keep the image, interaction history, and generated reasoning fixed, replace only the predicted type with the reference type, and let the same frozen student regenerate the parameters\. The student receives no teacher output or reference parameters\. Restoring the original type reproduces all136136original responses, confirming that the intervention changes only the type\.

## Appendix BEvaluation Metrics and Statistical Tests

##### Structured\-action diagnostics\.

The primary benchmark metric is task\-level SR\. To locate action errors, we additionally report action\-type accuracy and parameter acceptance conditional on a correct type\. Conditional parameter results are grouped by the reference parameter class and include the denominator of every class\. These metrics diagnose the source of task failures but do not replace SR\.

For the disagreement\-conditioned spatial\-action analysis, we report single\-step success rate\. Each sample contains one reference spatial action\. A prediction is a single\-step success only when it has the correct action type, a parseable coordinate, anddnorm≤0\.07d\_\{\\text\{norm\}\}\\leq 0\.07\. Wrong\-type and unparseable predictions are failures\. We compute this rate over the fixed spatial\-action subset and report its denominator and success count\. It is distinct from benchmark task\-level SR\.

## Appendix CAction\-Span Token and Signal Allocation

We draw300300responses from each domain and locate the final structured action using the same token boundaries used by the training masks\. For each response, we divide the number of action tokens by the number of valid response tokens\. The action accounts for4\.1%4\.1\\%of MobileWorld tokens,7\.1%7\.1\\%of OSWorld tokens, and3\.9%3\.9\\%of WebVoyager tokens\.

The archived training logs retain token\-level distillation values by normalized response position\. Across domains, action tokens receives4\.0%4\.0\\%–7\.0%7\.0\\%of the total logged distillation signal \(Table[7](https://arxiv.org/html/2607.29320#A3.T7)\), compared with13%13\\%–14%14\\%for the first ten response tokens\.

Table 7:Exact action\-token share and distillation signal share\.
## Appendix DAllocation Profile Breakdown

Table[8](https://arxiv.org/html/2607.29320#A4.T8)breaks the profile down by the reference parameter schema\. Coordinates dominate the sample; the URL subset has only five examples and supports no class\-specific conclusion\. These classes cover 841 examples, while 59 empty\-schema actions remain in the aggregate results in Table[3](https://arxiv.org/html/2607.29320#S4.T3)\. Empty\-schema actions require no parameters\. A type match is Correct, whereas a wrong is Type×\\times\. Param\.×\\timescannot occur\. Of these 59 actions, the initial student has 18 Correct and 41 Type×\\times, while the trained student has 44 Correct and 15 Type×\\times\.

Table 8:Allocation profile by reference parameter class\. C, T×\\times, and P×\\timesdenote correct, wrong\-type, and wrong\-parameter outcomes\.

相似文章

UI-MOPD:面向持续GUI智能体学习的多平台在线策略蒸馏

Hugging Face Daily Papers

本文提出UI-MOPD方法,将多教师在线策略蒸馏与持续学习相结合,用于跨平台训练GUI智能体,同时介绍了Uni-GUI数据集。该方法在OSWorld和MobileWorld上分别达到38.2%和12.0%的任务成功率,展示了有效的跨平台能力保持与适应。

面向GUI代理的技能引导连续蒸馏

arXiv cs.AI

该论文提出了技能引导连续蒸馏(SGCD),这是一个迭代式自我改进框架,利用技能引导策略在闭环执行过程中为偏离轨迹的状态生成监督信号,将OSWorld-Verified上GUI代理的成功率从约30%提升至超过50%。