Reinforcement Learning for Computer-Use Agents with Autonomous Evaluation

# Reinforcement Learning for Computer-Use Agents with Autonomous EvaluationAccepted to the 4th International Workshop on Generalizing from Limited Resources in the Open World (GLOW @ IJCAI 2026): https://glow-ijcai-2026.github.io/glow-ijcai-2026/.
Source: [https://arxiv.org/html/2606.24515](https://arxiv.org/html/2606.24515)
Oleksandr Kosovan1 1Ukrainian Catholic University, Lviv, Ukraine sumyk\.pn@ucu\.edu\.ua, o\.kosovan@ucu\.edu\.ua

###### Abstract

Computer\-Use Agents \(CUAs\) execute high\-level user goals by perceiving and acting directly within graphical user interfaces\. However, reinforcement learning for CUAs remains difficult because open\-ended desktop environments rarely provide scalable, machine\-readable reward signals: task success is often visually grounded and hard to specify with handcrafted reward functions or dense manual labels\.

We propose an RL fine\-tuning framework that uses autonomous vision\-language evaluation as a scalable supervision signal for GUI agents\. Given a final screenshot and the original instruction, a Vision\-Language Model judges task completion and provides terminal feedback without task\-specific heuristics or manual labels during policy optimization\.

Because autonomous evaluators are imperfect, we model their feedback as a noisy binary reward channel and derive a noise\-corrected reward estimator for Proximal Policy Optimization\. Experiments across macOSWorld, Windows Agent Arena, and OSWorld show that corrected evaluator rewards outperform both zero\-shot baselines and raw evaluator rewards, improving success rates by an average of 12\.6 percentage points over zero\-shot performance and 5\.1 points over raw evaluator fine\-tuning\. These results suggest that autonomous evaluation can serve as a practical reward signal for RL in GUI environments when evaluator noise is explicitly modeled and corrected\.

![Refer to caption](https://arxiv.org/html/2606.24515v1/Images/schema_CUA.png)Figure 1:Bias correction framework for autonomous evaluation rewards\.## 1Introduction

Computer\-Use Agents \(CUAs\) aim to autonomously operate graphical user interfaces \(GUIs\) from natural\-language instructions using visual observations and executing actionsLiuet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib1)\); Sunet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib3)\)\. Recent progress shows promising generalization across applications and operating systems, positioning CUAs as a foundation for service\-agnostic desktop automationWanget al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib4)\); Sageret al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib5)\)\. Despite these advances, current CUAs remain unreliable in unconstrained, real\-world settings\. Specifically, on the OSWorld benchmarkXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\), state\-of\-the\-art desktop agents achieve success rates of only around60%60\\%, highlighting a significant gap between current capabilities and practical deployment\. This gap motivates the need for more effective learning mechanisms to improve robustness and task success in real\-world desktop environments\.

Reinforcement learning \(RL\) provides a principled framework for improving agent behavior by enabling exploration, recovery from errors, and learning through interaction rather than reliance on static demonstrationsSutton and Barto \([2018](https://arxiv.org/html/2606.24515#bib.bib7)\)\. However, applying RL to GUI\-based environments remains fundamentally constrained by the absence of reliable reward signalsSumyk and Kosovan \([2025](https://arxiv.org/html/2606.24515#bib.bib17)\)\. Unlike robotics simulatorsTanget al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib8)\)or game environmentsKoyamadaet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib9)\), where success criteria are explicitly defined and machine\-readable, desktop applications rarely expose clear indicators of task completion\. Instead, task objectives are typically implicit, visually grounded, and highly unstructuredXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\)\.

As a result, prior work often relies on brittle, task\-specific heuristics, such as DOM element checks in web\-based environmentsQiet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib10)\), or on manually annotated success labelsXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\)\. These approaches fail to scale across different applications, operating systems, and task distributions, and they substantially limit generalization beyond narrowly defined settings\. An alternative strategy is to define task success internally within the computer\-use agent itselfWanget al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib4)\)\. However, this introduces additional reliability concerns, as the agent’s own perception and reasoning are inherently imperfect and are precisely the components that RL aims to improve\. Consequently, the lack of a scalable and reliable reward mechanism remains a central bottleneck for effective RL fine\-tuning of computer\-use agents\.

A promising alternative is to employ autonomous evaluators that assess whether a task has been successfully completed\. Recent worksSumyk and Kosovan \([2025](https://arxiv.org/html/2606.24515#bib.bib17)\); Linet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib15)\); Sumyk and Kosovan \([2026](https://arxiv.org/html/2606.24515#bib.bib18)\); Rossetet al\.\([2026](https://arxiv.org/html/2606.24515#bib.bib16)\)demonstrate that evaluators based on Vision\-Language Models \(VLMs\) can achieve high accuracy and generalize across diverse applications\. However, these evaluators inevitably introduce noise: they may incorrectly label unsuccessful executions as completed \(false positives\) or fail to detect success \(false negatives\)\. Naively using evaluator outputs as reward signals can therefore introduce bias and lead to unstable reinforcement learningWanget al\.\([2020a](https://arxiv.org/html/2606.24515#bib.bib19)\); Liet al\.\([2024a](https://arxiv.org/html/2606.24515#bib.bib21)\)\.

In this paper, we address this challenge by introducing a principled framework for integrating autonomous evaluators into reinforcement learning for CUAs\. Rather than treating evaluator outputs as ground truth, we explicitly model evaluator noise and derive a statistically grounded, asymptotically unbiased reward estimator\. This formulation enables effective RL fine\-tuning under imperfect feedback, allowing CUAs to learn from large\-scale interaction data without manual labeling\.

This work investigates the following research questions:

- •RQ1:Can autonomous VLM\-based evaluation serve as a scalable reward signal for RL fine\-tuning of Computer\-Use Agents?
- •RQ2:Does modeling and correcting evaluator noise improve policy learning compared to using raw evaluator rewards?

To address these questions, we propose a principled framework that integrates autonomous evaluators into the reinforcement\-learning loop by explicitly modeling their error characteristics\. Our approach derives a statistically grounded, asymptotically unbiased reward estimator that corrects evaluator\-induced noise and enables stable policy optimization\. Empirically, we demonstrate that this framework yields improvements in robustness and task success across three operating systems, Windows, macOS, and Linux, without relying on manual annotations or task\-specific heuristics\. The implementation is available at[https://github\.com/martasumyk/rl\_with\_autonomous\_feedback](https://github.com/martasumyk/rl_with_autonomous_feedback)\.

## 2Related Work

### 2\.1Computer\-Use Agents

Computer\-Use Agents \(CUAs\) are end\-to\-end autonomous systems that complete natural\-language tasks by perceiving the rendered desktop GUI, typically via screenshots, and executing actions such as clicking, typing, scrolling, and draggingSageret al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib5)\)\. Recent CUA architectures combine vision\-language reasoning with explicit action grounding to plan and carry out long\-horizon workflows across diverse applications and operating systemsLiuet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib1)\); Sunet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib3)\); Wanget al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib4)\); Qinet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib2)\)\.

Unlike API\-based or function\-calling agents that require explicit service integrations, CUAs adopt a service\-agnostic interaction model: they perceive and manipulate software exclusively through its rendered interface\. This design enables interaction with arbitrary applications without bespoke engineering and allows a single agent to generalize across different software applications and operating systemsSunet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib3)\); Sageret al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib5)\)\. Consequently, CUAs are increasingly regarded as a promising foundation for general\-purpose computer automation\.

However, this generality introduces significant challenges for reasoning and verification\. Because CUAs rely solely on visual feedback, they are susceptible to silent or partial failures caused by unexpected interface states, asynchronous rendering, visual occlusions, or subtle distribution shifts in UI layoutsGuret al\.\([2023](https://arxiv.org/html/2606.24515#bib.bib11)\); Humphreyset al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib12)\); Liet al\.\([2024b](https://arxiv.org/html/2606.24515#bib.bib13)\)\. Furthermore, many real\-world tasks lack explicit, machine\-readable success criteria, making it difficult to reliably determine whether an agent has truly completed the intended objectiveSumyk and Kosovan \([2025](https://arxiv.org/html/2606.24515#bib.bib17)\)\.

Moreover, state\-of\-the\-art desktop agents achieve success rates of only around60%60\\%on OSWorldXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\)and approximately40%40\\%on macOSWorldYanget al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib29)\)\. This performance gap highlights the need for more robust training and fine\-tuning methods for CUAs\.

### 2\.2Training and Fine\-Tuning of Computer\-Use Agents

Training CUAs typically combines supervised learning with reinforcement learningLaiet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib31)\); Wanget al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib4)\)\. Many systems first use behavioral cloning to map visual observations and instructions to low\-level GUI actions from human demonstrations or scripted trajectoriesGuret al\.\([2023](https://arxiv.org/html/2606.24515#bib.bib11)\); Liet al\.\([2024b](https://arxiv.org/html/2606.24515#bib.bib13)\); Humphreyset al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib12)\)\. This approach is effective for short\-horizon action prediction and instruction following, but it often struggles with compounding errors and long\-horizon tasks that require recovery, exploration, and verificationHumphreyset al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib12)\)\.

To address these limitations, recent work explores RL\-based fine\-tuning to improve robustness and task success beyond demonstrationsLiuet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib1)\); Sunet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib3)\); Qinet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib2)\)\. In practice, RL pipelines either operate in environments where rewards can be programmatically defined, such as synthetic web interfacesGuret al\.\([2023](https://arxiv.org/html/2606.24515#bib.bib11)\); Humphreyset al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib12)\), or rely on task\- and platform\-specific heuristics, such as DOM parsing or string matching, that do not transfer to general desktop settingsXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\)\. More broadly, existing approaches remain limited by the dependence on a reliable task\-completion signal, which is rarely available in real\-world desktop GUIs and motivates research on autonomous evaluation\.

### 2\.3Autonomous Evaluation

Autonomous evaluation seeks to determine whether an agent has successfully completed a user’s instruction based solely on the observed GUI state, producing feedback suitable for both benchmarking and learningPanet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib14)\)\. In real\-world desktop environments, task success is rarely accompanied by explicit, machine\-readable signals, as objectives are often implicit and visually groundedSumyk and Kosovan \([2025](https://arxiv.org/html/2606.24515#bib.bib17)\)\. Consequently, many existing benchmarks and training pipelines rely on human verification of final states, which is costly, time\-consuming, and difficult to scale across diverse applications and operating systemsXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\)\.

Recent work proposes autonomous evaluators, typically based on Vision\-Language Models \(VLMs\), that assess task completion by jointly analyzing the final GUI state and the natural\-language instruction, and output a binary success judgmentSumyk and Kosovan \([2025](https://arxiv.org/html/2606.24515#bib.bib17)\); Linet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib15)\)\. These evaluators enable scalable, automated success labeling and can serve as reward or feedback providers for agent improvement\. However, their predictions are imperfect and can include false positives and false negatives\. Therefore, naively treating evaluator outputs as ground\-truth rewards can bias learning and destabilize policy optimizationWanget al\.\([2020a](https://arxiv.org/html/2606.24515#bib.bib19)\)\. This motivates methods that explicitly model evaluator error and incorporate noise\-aware reward correction, which is the focus of our approach\.

### 2\.4Reinforcement Learning with Noisy Feedback

A broad line of research studies reinforcement learning when the agent does not observe the true reward, but instead receives a noisy proxy produced by measurement artifacts, imperfect annotators, or automated verification systemsWanget al\.\([2020a](https://arxiv.org/html/2606.24515#bib.bib19)\); Caiet al\.\([2025a](https://arxiv.org/html/2606.24515#bib.bib22)\); Wanget al\.\([2020b](https://arxiv.org/html/2606.24515#bib.bib20)\)\.

Wang et al\.Wanget al\.\([2020a](https://arxiv.org/html/2606.24515#bib.bib19)\)formalize reward corruption using a confusion matrix over discrete reward values\. They show that, given an estimate of the corruption process, one can construct an unbiased reward estimator\. This allows standard RL algorithms to recover optimal policies despite observing only corrupted feedback\.

Complementary work investigates learning from noisy evaluative signals provided by humans or teachers\. For example, CANDERE\-COACHLiet al\.\([2024a](https://arxiv.org/html/2606.24515#bib.bib21)\)considers unreliable binary approve/disapprove feedback and proposes online denoising mechanisms that filter feedback before policy updates, demonstrating robustness under substantial noise\.

Most closely related to our setting, Cai et al\.Caiet al\.\([2025b](https://arxiv.org/html/2606.24515#bib.bib23)\)study reinforcement learning with verifiable rewards, where policies are trained from noisy binary signals generated by automated verifiers\. They model asymmetric false\-positive and false\-negative errors and derive correction strategies that debias policy\-gradient updates by appropriately transforming the observed feedback\.

Our method instantiates these ideas for computer\-use agents by treating a vision\-based task\-completion judge as a noisy binary reward channel\. Following prior workWanget al\.\([2020a](https://arxiv.org/html/2606.24515#bib.bib19)\); Caiet al\.\([2025b](https://arxiv.org/html/2606.24515#bib.bib23)\), we derive a simple correction that yields an asymptotically unbiased reward estimator under a mild separability condition and integrate it directly into PPO\. Unlike approaches that denoise feedback at the data levelLiet al\.\([2024a](https://arxiv.org/html/2606.24515#bib.bib21)\), our correction operates at the reward level, making it straightforward to plug into standard policy\-gradient fine\-tuning\.

## 3Methodology

We formulate a CUA as a Markov Decision Processℳ=\(𝒮,𝒜,P,r,γ\)\\mathcal\{M\}=\(\\mathcal\{S\},\\mathcal\{A\},P,r,\\gamma\)van Otterlo and Wiering \([2012](https://arxiv.org/html/2606.24515#bib.bib32)\)\. We consider a collection ofmmtasks indexed byi∈\{1,…,m\}i\\in\\\{1,\\dots,m\\\}, where each task defines an episode with a fixed natural\-language instructiondid\_\{i\}and horizonTT\.

At timestepttof taskii, the agent observes the current GUI and executes a single atomic interaction\. The state is defined assi,t=\(xi,t,di\)s\_\{i,t\}=\(x\_\{i,t\},d\_\{i\}\), wherexi,tx\_\{i,t\}denotes the rendered screen image anddid\_\{i\}the task description, fixed throughout the episode\. The action space𝒜\\mathcal\{A\}consists of low\-level GUI operations:click,type,scroll, anddrag\. A trajectory for taskiiis:

τi=\(si,0,ai,0,…,si,T\)\.\\tau\_\{i\}=\(s\_\{i,0\},a\_\{i,0\},\\dots,s\_\{i,T\}\)\.\(1\)
### 3\.1Dataset

We construct a synthetic dataset of computer\-use tasks spanning4242applications for each OS, including built\-in macOS applications and functionally similar applications on Linux and Windows\. For each application, we define6060natural\-language task descriptions, resulting in a total of7,5607\{,\}560tasks that cover a broad range of GUI interaction patterns across operating systems\.

For each application, tasks are randomly partitioned into three disjoint splits of2020tasks each\. Two splits, a total of4040tasks per application and two\-thirds of the dataset, are used for reinforcement\-learning fine\-tuning, where agent rollouts are collected and PPO updates are performed using evaluator\-based rewards\. The remaining split, one\-third of the dataset, is reserved for evaluator calibration and is used to estimate the evaluator’s true\-positive, false\-positive, true\-negative, and false\-negative rates\. This separation ensures that evaluator calibration and policy optimization are conducted on disjoint task sets, preventing information leakage\.

In addition to our dataset, we incorporate task descriptions from three existing GUI interaction datasets, OmniActKapooret al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib24)\), GUI\-WorldChenet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib25)\), and GUIDEChawlaet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib26)\), to further diversify the task distribution used for RL fine\-tuning\. From these datasets, we use only the natural\-language task specifications as input to the agent\. A summary of all datasets and their task characteristics is provided in Table[1](https://arxiv.org/html/2606.24515#S3.T1)\.

In addition to task definitions, the dataset includes execution logs of a base computer\-use agent, UI\-TARSQinet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib2)\), and of agents fine\-tuned with reinforcement learning\. Each log records a complete agent trajectory, including screenshots, executed actions, and intermediate reasoning steps\. For evaluation purposes, task outcomes in the evaluator calibration and RL test splits are annotated with ground\-truth success labels, which are used exclusively for evaluator assessment and final benchmarking and are never exposed to the agent during training\.

Table 1:Summary of datasets whose task descriptions are used as input for RL fine\-tuning\. All datasets provide natural\-language task specifications for GUI interaction; our dataset additionally includes native desktop tasks explicitly designed for autonomous evaluation and reward correction\.
### 3\.2Autonomous Evaluation

For autonomous evaluation, we use the Qwen2\-VL\-7B modelWanget al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib27)\), motivated by its strong multimodal reasoning and empirically demonstrated superiority as a vision\-based evaluator among open\-source modelsSumyk and Kosovan \([2025](https://arxiv.org/html/2606.24515#bib.bib17)\); Linet al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib15)\)\. Given a task descriptiondid\_\{i\}and the final GUI statexi,Tx\_\{i,T\}, the evaluator outputs a binary signal:

r~i∈\{0,1\},\\tilde\{r\}\_\{i\}\\in\\\{0,1\\\},\(2\)wherer~i=1\\tilde\{r\}\_\{i\}=1indicates that the task is judged completed\. The evaluator operates in a zero\-shot setting and is fully decoupled from the acting agent: it observes neither the agent’s action history nor its internal reasoning, basing its judgment solely on the final screenshot and instruction\.

### 3\.3Reinforcement Learning Fine\-Tuning

We now describe how we fine\-tune a pre\-trained CUA using reinforcement learning with world feedback from the autonomous evaluator\. Figure[2](https://arxiv.org/html/2606.24515#S3.F2)provides an overview of the full pipeline\.

#### Terminal Reward and Noise Model\.

In our setting, reward is terminal\-only: the true task\-completion signalri⋆∈\{0,1\}r\_\{i\}^\{\\star\}\\in\\\{0,1\\\}is defined only at the end of the episode and is unobserved\. Instead, the evaluator provides a noisy binary judgmentr~i\\tilde\{r\}\_\{i\}based on the final GUI statexi,Tx\_\{i,T\}\.

We characterize evaluator noise using conditional error rates:

e\+:=Pr⁡\(r~i=0∣ri⋆=1\)\(false negative\),e\_\{\+\}:=\\Pr\(\\tilde\{r\}\_\{i\}=0\\mid r\_\{i\}^\{\\star\}=1\)\\quad\\text\{\(false negative\)\},\(3\)e−:=Pr⁡\(r~i=1∣ri⋆=0\)\(false positive\)\.e\_\{\-\}:=\\Pr\(\\tilde\{r\}\_\{i\}=1\\mid r\_\{i\}^\{\\star\}=0\)\\quad\\text\{\(false positive\)\}\.\(4\)
We define the separability constant:

D:=1−e\+−e−,D:=1\-e\_\{\+\}\-e\_\{\-\},\(5\)and assumeD\>0D\>0, ensuring that the evaluator provides informative feedback beyond random guessing\.

#### Noise\-Corrected Reward Estimator\.

Sincer~i\\tilde\{r\}\_\{i\}is binary, its conditional expectation satisfies:

𝔼​\[r~i∣ri⋆\]=e−\+D⋅ri⋆\.\\mathbb\{E\}\[\\tilde\{r\}\_\{i\}\\mid r\_\{i\}^\{\\star\}\]=e\_\{\-\}\+D\\cdot r\_\{i\}^\{\\star\}\.\(6\)Solving forri⋆r\_\{i\}^\{\\star\}yields the corrected reward estimator:

r^i:=r~i−e−D,\\hat\{r\}\_\{i\}:=\\frac\{\\tilde\{r\}\_\{i\}\-e\_\{\-\}\}\{D\},\(7\)which satisfies:

𝔼​\[r^i∣τi\]=ri⋆,\\mathbb\{E\}\[\\hat\{r\}\_\{i\}\\mid\\tau\_\{i\}\]=r\_\{i\}^\{\\star\},\(8\)and is therefore an asymptotically unbiased estimator of true task completion\.

In practice, error ratese\+e\_\{\+\}ande−e\_\{\-\}are unknown and must be estimated from the held\-out calibration split\. Lete^\+\\hat\{e\}\_\{\+\}ande^−\\hat\{e\}\_\{\-\}denote empirical estimates andD^:=1−e^\+−e^−\\hat\{D\}:=1\-\\hat\{e\}\_\{\+\}\-\\hat\{e\}\_\{\-\}\. The resulting plug\-in estimator is:

r^i♯:=r~i−e^−D^,\\hat\{r\}\_\{i\}^\{\\,\\sharp\}:=\\frac\{\\tilde\{r\}\_\{i\}\-\\hat\{e\}\_\{\-\}\}\{\\hat\{D\}\},\(9\)which is asymptotically unbiased as calibration data grows andD^\\hat\{D\}remains bounded away from zero\.

#### Policy Optimization\.

Since reward is terminal\-only, the discounted return for taskiiis:

R^i:=γT​r^i,\\hat\{R\}\_\{i\}:=\\gamma^\{T\}\\,\\hat\{r\}\_\{i\},\(10\)and the RL objective is:

J​\(θ\):=𝔼i∼𝒟,τi∼πθ​\[R^i\]\.J\(\\theta\):=\\mathbb\{E\}\_\{i\\sim\\mathcal\{D\},\\,\\tau\_\{i\}\\sim\\pi\_\{\\theta\}\}\\\!\\left\[\\hat\{R\}\_\{i\}\\right\]\.\(11\)
We optimizeJ​\(θ\)J\(\\theta\)using Proximal Policy Optimization \(PPO\)Schulmanet al\.\([2017](https://arxiv.org/html/2606.24515#bib.bib28)\)\. The actor parameterizes a stochastic policyπθ​\(a∣s\)\\pi\_\{\\theta\}\(a\\mid s\)over GUI actions; the critic estimates the value functionVϕ​\(s\)V\_\{\\phi\}\(s\)\. Advantage estimates are computed by propagating the terminal corrected rewardr^i\\hat\{r\}\_\{i\}backward through the trajectory, enabling stable policy\-gradient updates despite evaluator\-induced noise\. By explicitly correcting evaluator errors, our approach provides a reliable world feedback signal for RL without requiring ground\-truth labels or task\-specific heuristics\.

![Refer to caption](https://arxiv.org/html/2606.24515v1/x1.png)Figure 2:Overview of our RL fine\-tuning pipeline with autonomous, noise\-corrected rewards\. For each task instance, the computer\-use agent interacts with the desktop environment, observing the GUI statests\_\{t\}and executing actionsata\_\{t\}\. A vision\-language evaluator predicts a noisy binary completion signalr~\\tilde\{r\}from the final observed state, a screenshot and instruction, which we transform into a corrected rewardr^=f​\(r~\)\\hat\{r\}=f\(\\tilde\{r\}\)using estimated false\-positive and false\-negative rates\. PPO then updates the agent parametersθ\\thetausingr^\\hat\{r\}, closing the interaction–evaluation–learning loop\.

## 4Results and Evaluation

We evaluate our approach along three dimensions: \(i\) the accuracy and error profile of the autonomous evaluator used to generate reward signals; \(ii\) the effect of evaluator\-based rewards on PPO fine\-tuning, comparing raw evaluator feedback with the proposed noise\-corrected reward estimator; and \(iii\) an ablation comparing per\-OS fine\-tuning against a single unified model trained across operating systems\.

### 4\.1Autonomous Evaluation

We first quantify the reliability of the vision\-based evaluator, since its error rates directly determine the reward correction in Section[3](https://arxiv.org/html/2606.24515#S3)\. Table[2](https://arxiv.org/html/2606.24515#S4.T2)reports normalized confusion matrices on the held\-out evaluator split for macOS, Windows, and Linux\. While the evaluator achieves high accuracy overall, its behavior varies by OS\.

These asymmetries are important for reinforcement learning: false positives are particularly harmful because they can assign high reward to unsuccessful trajectories, reinforcing incorrect behaviors; false negatives, in contrast, primarily reduce the effective reward frequency and can slow learning\. We therefore estimated\(e\+,e−\)\(e\_\{\+\},e\_\{\-\}\)per OS from the evaluator split and used these values to compute the corrected reward estimatorr^t\\hat\{r\}\_\{t\}during PPO fine\-tuning\.

Table 2:Normalized confusion matrices of the autonomous evaluator across operating systems\. Green cells indicate correct predictions, red cells indicate misclassifications\.
### 4\.2Reinforcement Learning Fine\-Tuning

We evaluate reinforcement learning fine\-tuning across three desktop environments: macOS, Windows, and Linux, using macOSWorldYanget al\.\([2025](https://arxiv.org/html/2606.24515#bib.bib29)\), Windows Agent ArenaBonattiet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib30)\), and OSWorldXieet al\.\([2024](https://arxiv.org/html/2606.24515#bib.bib6)\), respectively\. Results are summarized in Table[3](https://arxiv.org/html/2606.24515#S4.T3)\.

We compare five training configurations: \(1\) a zero\-shot baseline, \(2\) PPO fine\-tuning with raw binary evaluator rewardsr~t\\tilde\{r\}\_\{t\}using a unified cross\-OS model, \(3\) PPO fine\-tuning with the corrected reward estimatorr^t\\hat\{r\}\_\{t\}using a unified model, \(4\) PPO fine\-tuning with raw evaluator rewards using per\-OS models, and \(5\) PPO fine\-tuning with the corrected reward estimator using per\-OS models\.

Table 3:Task success rates across desktop benchmarks\. We compare the zero\-shot baseline with PPO fine\-tuning using \(i\) raw binary evaluator rewards and \(ii\) the proposed noise\-corrected reward estimator, under both per\-OS and unified cross\-OS training\. Best results are shown inbold, second\-best are underlined\.Across all benchmarks, reinforcement learning with evaluator feedback improves performance over the zero\-shot baseline\. However, the choice of reward signal and model parameterization substantially affects both final performance and generalization\. Using raw evaluator rewards yields moderate gains in most settings, but these gains are consistently smaller than those obtained with the corrected reward estimator\.

The proposed noise\-corrected rewardr^t\\hat\{r\}\_\{t\}achieves the strongest performance across all three benchmarks when combined with per\-OS fine\-tuning, improving success rates from0\.084→0\.2030\.084\\rightarrow 0\.203on macOSWorld,0\.331→0\.4420\.331\\rightarrow 0\.442on Windows Agent Arena, and0\.283→0\.4320\.283\\rightarrow 0\.432on OSWorld\. These improvements demonstrate that explicitly accounting for evaluator false positives and false negatives yields a substantially more reliable learning signal for PPO\.

In contrast, fine\-tuning with raw evaluator rewards exhibits inconsistent behavior\. While raw rewards can improve in\-domain performance, they underperform the corrected estimator in all settings and, in some cases, reduce generalization\. This effect is most evident on OSWorld, where the corrected reward consistently outperforms raw rewards under both unified and per\-OS training\. These results indicate that ignoring evaluator error can introduce bias that negatively impacts reinforcement learning, particularly under distribution shift\.

## 5Discussion and Limitations

Firstly, our approach assumes that the evaluator’s false\-negative and false\-positive rates\(e\+,e−\)\(e\_\{\+\},e\_\{\-\}\)are approximately fixed for a given model and evaluation protocol and can be reliably estimated on a held\-out calibration split\. In practice, evaluator behavior may be non\-stationary: error rates can vary with task type, UI complexity, language phrasing, visual themes, and operating\-system\-specific interface conventions\. Moreover, as the agent improves during training, the distribution of visited GUI states may shift, potentially changing the evaluator’s error profile and introducing residual bias when using plug\-in estimates\(e^\+,e^−\)\(\\hat\{e\}\_\{\+\},\\hat\{e\}\_\{\-\}\)\. A natural extension is to estimate error rates conditionally, for example, per task family or application type\.

Another limitation is that our formulation relies on a binary success signal, which is inherently sparse and often effectively terminal\. While PPO can learn under sparse rewards, exploration remains challenging for long\-horizon GUI tasks with delayed success\. The proposed correction improves reward accuracy but does not increase reward density, and therefore cannot by itself resolve exploration difficulties\. Incorporating shaping signals, such as intermediate evaluator judgments, progress estimators, or subgoal completion signals, may substantially improve sample efficiency, but would require additional calibration and may introduce new sources of bias\.

The effectiveness of the correction also depends on the quality of the calibration set used to estimate\(e\+,e−\)\(e\_\{\+\},e\_\{\-\}\)\. With limited calibration data, statistical uncertainty in these estimates can propagate into the corrected reward and increase the variance of policy\-gradient updates, particularly when the separability termD=1−e\+−e−D=1\-e\_\{\+\}\-e\_\{\-\}is small and the correction amplifies noise\. In such regimes, conservative strategies such as clipping the corrected reward, shrinking estimates toward priors, or abstaining when evaluator confidence is low may be necessary to maintain training stability\.

In addition, the corrected rewardr^t=\(r~t−e^−\)D^\\hat\{r\}\_\{t\}=\\frac\{\(\\tilde\{r\}\_\{t\}\-\\hat\{e\}\_\{\-\}\)\}\{\\hat\{D\}\}is real\-valued and can fall outside the\[0,1\]\[0,1\]range, which alters the scale of advantages and may affect PPO optimization in finite\-sample settings\. Although policy\-gradient methods are invariant to affine reward transformations in expectation, practical stability can still depend on careful normalization and hyperparameter choices\.

Finally, vision\-based evaluators have inherent blind spots: they may miss semantically correct outcomes that are visually subtle or overestimate success based on superficial visual cues\. Binary success labels also ignore partial progress and alternative valid solutions, which are common in real\-world GUI tasks\. Extending evaluators to produce calibrated confidence scores, multi\-level outcomes, or preference\-based feedback could provide richer learning signals, but would require revisiting both the noise model and the correction mechanism\. While our results demonstrate improved generalization to OSWorld, robustness under broader distribution shifts, such as new application versions, different screen resolutions, or accessibility settings, remains an open direction for future work\.

## 6Conclusion

We presented a principled approach for reinforcement\-learning fine\-tuning of computer\-use agents using reward signals produced by an autonomous vision\-based evaluator\. Our key idea is to treat the evaluator as a binary reward channel and to correct its false\-positive and false\-negative errors via a simple reward transformation that is asymptotically unbiased under a mild separability condition\. This yields a drop\-in replacement for raw evaluator rewards that can be used with standard policy\-gradient methods such as PPO\.

Empirically, PPO fine\-tuning with evaluator feedback improves over the zero\-shot UI\-TARS baseline across operating systems\. While fine\-tuning with the raw evaluator signal can help on in\-domain tasks, we find that explicitly correcting evaluator noise produces more consistent gains, especially on macOS and Windows, and improves transfer to the OSWorld benchmark, where naive use of raw rewards slightly degrades performance\. Finally, our ablation comparing per\-OS and unified fine\-tuning suggests that OS\-specific policies provide more reliable generalization overall, supporting the use of per\-OS models in our main experiments\.

Overall, these results indicate that autonomous evaluation can serve as a scalable supervision signal for RL in GUI environments, provided that evaluator noise is explicitly modeled and corrected\. We view this as a step toward practical RL fine\-tuning pipelines for general desktop agents that learn from interaction without requiring manual success labels\.

## References

- R\. Bonatti, D\. Zhao, F\. Bonacci, D\. Dupont, S\. Abdali, Y\. Li, Y\. Lu, J\. Wagle, K\. Koishida, A\. Bucker, L\. Jang, and Z\. Hui \(2024\)Windows agent arena: evaluating multi\-modal os agents at scale\.External Links:2409\.08264,[Link](https://arxiv.org/abs/2409.08264)Cited by:[§4\.2](https://arxiv.org/html/2606.24515#S4.SS2.p1.1)\.
- X\. Cai, W\. Wang, F\. Liu, T\. Liu, G\. Niu, and M\. Sugiyama \(2025a\)Reinforcement learning with verifiable yet noisy rewards under imperfect verifiers\.External Links:2510\.00915,[Link](https://arxiv.org/abs/2510.00915)Cited by:[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p1.1)\.
- X\. Cai, W\. Wang, F\. Liu, T\. Liu, G\. Niu, and M\. Sugiyama \(2025b\)Reinforcement learning with verifiable yet noisy rewards under imperfect verifiers\.arXiv preprint arXiv:2510\.00915\.External Links:[Link](https://arxiv.org/abs/2510.00915)Cited by:[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p4.1),[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p5.1)\.
- R\. Chawla, A\. Jha, M\. Kumar, M\. NS, and I\. Bhola \(2024\)GUIDE: graphical user interface data for execution\.External Links:2404\.16048,[Link](https://arxiv.org/abs/2404.16048)Cited by:[§3\.1](https://arxiv.org/html/2606.24515#S3.SS1.p3.1),[Table 1](https://arxiv.org/html/2606.24515#S3.T1.2.4.3.1)\.
- D\. Chen, Y\. Huang, S\. Wu, J\. Tang, L\. Chen, Y\. Bai, Z\. He, C\. Wang, H\. Zhou, Y\. Li, T\. Zhou, Y\. Yu, C\. Gao, Q\. Zhang, Y\. Gui, Z\. Li, Y\. Wan, P\. Zhou, J\. Gao, and L\. Sun \(2025\)GUI\-world: a video benchmark and dataset for multimodal gui\-oriented understanding\.External Links:2406\.10819,[Link](https://arxiv.org/abs/2406.10819)Cited by:[§3\.1](https://arxiv.org/html/2606.24515#S3.SS1.p3.1),[Table 1](https://arxiv.org/html/2606.24515#S3.T1.2.3.2.1)\.
- I\. Gur, A\. Pal, T\. Li, M\. Brockschmidt, S\. Chaudhuri, M\. Riedl, and J\. Andreas \(2023\)BrowserGym: a benchmark for browser agents\.arXiv preprint arXiv:2307\.04492\.External Links:[Link](https://arxiv.org/abs/2307.04492)Cited by:[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p3.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p2.1)\.
- P\. Humphreys, A\. Ni, H\. Pan, I\. Gur, V\. Zhong, and J\. Andreas \(2024\)WebArena: a realistic web environment for building autonomous agents\.arXiv preprint arXiv:2307\.13854\.External Links:[Link](https://arxiv.org/abs/2307.13854)Cited by:[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p3.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p1.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p2.1)\.
- R\. Kapoor, Y\. P\. Butala, M\. Russak, J\. Y\. Koh, K\. Kamble, W\. Alshikh, and R\. Salakhutdinov \(2024\)OmniACT: a dataset and benchmark for enabling multimodal generalist autonomous agents for desktop and web\.External Links:2402\.17553,[Link](https://arxiv.org/abs/2402.17553)Cited by:[§3\.1](https://arxiv.org/html/2606.24515#S3.SS1.p3.1),[Table 1](https://arxiv.org/html/2606.24515#S3.T1.2.2.1.1)\.
- S\. Koyamada, S\. Okano, S\. Nishimori, Y\. Murata, K\. Habara, H\. Kita, and S\. Ishii \(2024\)Pgx: hardware\-accelerated parallel game simulators for reinforcement learning\.External Links:2303\.17503,[Link](https://arxiv.org/abs/2303.17503)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p2.1)\.
- H\. Lai, X\. Liu, Y\. Zhao, H\. Xu, H\. Zhang, B\. Jing, Y\. Ren, S\. Yao, Y\. Dong, and J\. Tang \(2025\)ComputerRL: scaling end\-to\-end online reinforcement learning for computer use agents\.External Links:2508\.14040,[Link](https://arxiv.org/abs/2508.14040)Cited by:[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p1.1)\.
- Y\. Li, S\. Das, and M\. E\. Taylor \(2024a\)CANDERE\-coach: reinforcement learning from noisy feedback\.arXiv preprint arXiv:2409\.15521\.External Links:[Link](https://arxiv.org/abs/2409.15521)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p4.1),[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p3.1),[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p5.1)\.
- Z\. Li, Y\. Zhao, Q\. Chen, Y\. Zhao, H\. Zhang, L\. Yuan, B\. Y\. Lin, Y\. Wang, and W\. Zhang \(2024b\)SeeAct: a multi\-modal agent for web interaction via visual grounding and action generation\.arXiv preprint arXiv:2404\.05719\.External Links:[Link](https://arxiv.org/abs/2404.05719)Cited by:[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p3.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p1.1)\.
- H\. Lin, X\. Tan, Y\. Qin, Z\. Xu, Y\. Shi, Z\. Li, G\. Li, S\. Cai, S\. Cai, C\. Fu, K\. Li, and X\. Sun \(2025\)CUARewardBench: a benchmark for evaluating reward models on computer\-using agent\.External Links:2510\.18596,[Link](https://arxiv.org/abs/2510.18596)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p4.1),[§2\.3](https://arxiv.org/html/2606.24515#S2.SS3.p2.1),[§3\.2](https://arxiv.org/html/2606.24515#S3.SS2.p1.2)\.
- Y\. Liu, P\. Li, Z\. Wei, C\. Xie, X\. Hu, X\. Xu, S\. Zhang, X\. Han, H\. Yang, and F\. Wu \(2025\)InfiGUIAgent: a multimodal generalist gui agent with native reasoning and reflection\.External Links:2501\.04575,[Link](https://arxiv.org/abs/2501.04575)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p2.1)\.
- J\. Pan, Y\. Zhang, N\. Tomlin, Y\. Zhou, S\. Levine, and A\. Suhr \(2024\)Autonomous evaluation and refinement of digital agents\.External Links:2404\.06474,[Link](https://arxiv.org/abs/2404.06474)Cited by:[§2\.3](https://arxiv.org/html/2606.24515#S2.SS3.p1.1)\.
- Z\. Qi, X\. Liu, I\. L\. Iong, H\. Lai, X\. Sun, W\. Zhao, Y\. Yang, X\. Yang, J\. Sun, S\. Yao, T\. Zhang, W\. Xu, J\. Tang, and Y\. Dong \(2025\)WebRL: training llm web agents via self\-evolving online curriculum reinforcement learning\.External Links:2411\.02337,[Link](https://arxiv.org/abs/2411.02337)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p3.1)\.
- Y\. Qin, Y\. Ye, J\. Fang, H\. Wang, S\. Liang, S\. Tian, J\. Zhang, J\. Li, Y\. Li, S\. Huang, W\. Zhong, K\. Li, J\. Yang, Y\. Miao, W\. Lin, L\. Liu, X\. Jiang, Q\. Ma, J\. Li, X\. Xiao, K\. Cai, C\. Li, Y\. Zheng, C\. Jin, C\. Li, X\. Zhou, M\. Wang, H\. Chen, Z\. Li, H\. Yang, H\. Liu, F\. Lin, T\. Peng, X\. Liu, and G\. Shi \(2025\)UI\-tars: pioneering automated gui interaction with native agents\.External Links:2501\.12326,[Link](https://arxiv.org/abs/2501.12326)Cited by:[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p2.1),[§3\.1](https://arxiv.org/html/2606.24515#S3.SS1.p4.1)\.
- C\. Rosset, P\. Sharma, A\. Zhao, M\. Gonzalez\-Fernandez, and A\. Awadallah \(2026\)The art of building verifiers for computer use agents\.External Links:2604\.06240,[Link](https://arxiv.org/abs/2604.06240)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p4.1)\.
- P\. J\. Sager, B\. Meyer, P\. Yan, R\. von Wartburg\-Kottler, L\. Etaiwi, A\. Enayati, G\. Nobel, A\. Abdulkadir, B\. F\. Grewe, and T\. Stadelmann \(2025\)A comprehensive survey of agents for computer use: foundations, challenges, and future directions\.External Links:2501\.16150,[Link](https://arxiv.org/abs/2501.16150)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p2.1)\.
- J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. Klimov \(2017\)Proximal policy optimization algorithms\.External Links:1707\.06347,[Link](https://arxiv.org/abs/1707.06347)Cited by:[§3\.3](https://arxiv.org/html/2606.24515#S3.SS3.SSS0.Px3.p2.4)\.
- M\. Sumyk and O\. Kosovan \(2025\)“Are we done yet?”: a vision\-based judge for autonomous task completion of computer use agents\.External Links:2511\.20067,[Link](https://arxiv.org/abs/2511.20067)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p2.1),[§1](https://arxiv.org/html/2606.24515#S1.p4.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p3.1),[§2\.3](https://arxiv.org/html/2606.24515#S2.SS3.p1.1),[§2\.3](https://arxiv.org/html/2606.24515#S2.SS3.p2.1),[§3\.2](https://arxiv.org/html/2606.24515#S3.SS2.p1.2)\.
- M\. Sumyk and O\. Kosovan \(2026\)CUAAudit: meta\-evaluation of vision\-language models as auditors of autonomous computer\-use agents\.External Links:2603\.10577,[Link](https://arxiv.org/abs/2603.10577)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p4.1)\.
- Z\. Sun, Z\. Liu, Y\. Zang, Y\. Cao, X\. Dong, T\. Wu, D\. Lin, and J\. Wang \(2025\)SEAgent: self\-evolving computer use agent with autonomous learning from experience\.External Links:2508\.04700,[Link](https://arxiv.org/abs/2508.04700)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p1.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p1.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p2.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p2.1)\.
- R\. S\. Sutton and A\. G\. Barto \(2018\)Reinforcement learning: an introduction\.2 edition,MIT Press,Cambridge, MA\.Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p2.1)\.
- C\. Tang, B\. Abbatematteo, J\. Hu, R\. Chandra, R\. Martín\-Martín, and P\. Stone \(2024\)Deep reinforcement learning for robotics: a survey of real\-world successes\.External Links:2408\.03539,[Link](https://arxiv.org/abs/2408.03539)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p2.1)\.
- M\. van Otterlo and M\. Wiering \(2012\)Reinforcement learning and markov decision processes\.Reinforcement Learning: State of the Art,pp\. 3–42\.External Links:[Document](https://dx.doi.org/10.1007/978-3-642-27645-3%5F1)Cited by:[§3](https://arxiv.org/html/2606.24515#S3.p1.5)\.
- H\. Wang, H\. Zou, H\. Song, J\. Feng, J\. Fang, J\. Lu, L\. Liu, Q\. Luo, S\. Liang, S\. Huang, W\. Zhong, Y\. Ye, Y\. Qin, Y\. Xiong, Y\. Song, Z\. Wu, A\. Li, B\. Li, C\. Dun, C\. Liu, D\. Zan, F\. Leng, H\. Wang, H\. Yu, H\. Chen, H\. Guo, J\. Su, J\. Huang, K\. Shen, K\. Shi, L\. Yan, P\. Zhao, P\. Liu, Q\. Ye, R\. Zheng, S\. Xin, W\. X\. Zhao, W\. Heng, W\. Huang, W\. Wang, X\. Qin, Y\. Lin, Y\. Wu, Z\. Chen, Z\. Wang, B\. Zhong, X\. Zhang, X\. Li, Y\. Li, Z\. Zhao, C\. Jiang, F\. Wu, H\. Zhou, J\. Pang, L\. Han, Q\. Liu, Q\. Ma, S\. Liu, S\. Cai, W\. Fu, X\. Liu, Y\. Wang, Z\. Zhang, B\. Zhou, G\. Li, J\. Shi, J\. Yang, J\. Tang, L\. Li, Q\. Han, T\. Lu, W\. Lin, X\. Tong, X\. Li, Y\. Zhang, Y\. Miao, Z\. Jiang, Z\. Li, Z\. Zhao, C\. Li, D\. Ma, F\. Lin, G\. Zhang, H\. Yang, H\. Guo, H\. Zhu, J\. Liu, J\. Du, K\. Cai, K\. Li, L\. Yuan, M\. Han, M\. Wang, S\. Guo, T\. Cheng, X\. Ma, X\. Xiao, X\. Huang, X\. Chen, Y\. Du, Y\. Chen, Y\. Wang, Z\. Li, Z\. Yang, Z\. Zeng, C\. Jin, C\. Li, H\. Chen, H\. Chen, J\. Chen, Q\. Zhao, and G\. Shi \(2025\)UI\-tars\-2 technical report: advancing gui agent with multi\-turn reinforcement learning\.External Links:2509\.02544,[Link](https://arxiv.org/abs/2509.02544)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p1.1),[§1](https://arxiv.org/html/2606.24515#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p1.1)\.
- J\. Wang, Y\. Liu, and B\. Li \(2020a\)Reinforcement learning with perturbed rewards\.arXiv preprint arXiv:1810\.01032\.External Links:[Link](https://arxiv.org/abs/1810.01032)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p4.1),[§2\.3](https://arxiv.org/html/2606.24515#S2.SS3.p2.1),[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p1.1),[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p2.1),[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p5.1)\.
- J\. Wang, Y\. Liu, and B\. Li \(2020b\)Reinforcement learning with perturbed rewards\.External Links:1810\.01032,[Link](https://arxiv.org/abs/1810.01032)Cited by:[§2\.4](https://arxiv.org/html/2606.24515#S2.SS4.p1.1)\.
- P\. Wang, S\. Bai, S\. Tan, S\. Wang, Z\. Fan, J\. Bai, K\. Chen, X\. Liu, J\. Wang, W\. Ge, Y\. Fan, K\. Dang, M\. Du, X\. Ren, R\. Men, D\. Liu, C\. Zhou, J\. Zhou, and J\. Lin \(2024\)Qwen2\-vl: enhancing vision\-language model’s perception of the world at any resolution\.External Links:2409\.12191,[Link](https://arxiv.org/abs/2409.12191)Cited by:[§3\.2](https://arxiv.org/html/2606.24515#S3.SS2.p1.2)\.
- T\. Xie, D\. Zhang, J\. Chen, X\. Li, S\. Zhao, R\. Cao, T\. J\. Hua, Z\. Cheng, D\. Shin, F\. Lei, Y\. Liu, Y\. Xu, S\. Zhou, S\. Savarese, C\. Xiong, V\. Zhong, and T\. Yu \(2024\)OSWorld: benchmarking multimodal agents for open\-ended tasks in real computer environments\.External Links:2404\.07972,[Link](https://arxiv.org/abs/2404.07972)Cited by:[§1](https://arxiv.org/html/2606.24515#S1.p1.1),[§1](https://arxiv.org/html/2606.24515#S1.p2.1),[§1](https://arxiv.org/html/2606.24515#S1.p3.1),[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p4.2),[§2\.2](https://arxiv.org/html/2606.24515#S2.SS2.p2.1),[§2\.3](https://arxiv.org/html/2606.24515#S2.SS3.p1.1),[§4\.2](https://arxiv.org/html/2606.24515#S4.SS2.p1.1)\.
- P\. Yang, H\. Ci, and M\. Z\. Shou \(2025\)MacOSWorld: a multilingual interactive benchmark for gui agents\.External Links:2506\.04135,[Link](https://arxiv.org/abs/2506.04135)Cited by:[§2\.1](https://arxiv.org/html/2606.24515#S2.SS1.p4.2),[§4\.2](https://arxiv.org/html/2606.24515#S4.SS2.p1.1)\.