基于高斯与Beta策略的图像级连续控制中的分歧正则化模仿学习

# Disagreement-Regularized Imitation Learning for Image-Based Continuous Control with Gaussian and Beta Policies
Source: [https://arxiv.org/html/2609.38407](https://arxiv.org/html/2609.38407)
Irving Giovani Bronzatti PetrazziniEmail:[irving\.petrazzini@posgrad\.ufsc\.br](mailto:[email protected])Affiliation:Department of Automation and Systems Engineering, Federal University of Santa Catarina, Florianópolis, Santa Catarina, BrazilEric Aislan AntoneloEmail:[eric\.antonelo@ufsc\.br](mailto:[email protected])Affiliation:Department of Automation and Systems Engineering, Federal University of Santa Catarina, Florianópolis, Santa Catarina, Brazil

###### Abstract

Purpose:Behavior cloning can accumulate errors when a learned controller visits states outside the demonstrated distribution\. This study evaluates whether Disagreement\-Regularized Imitation Learning \(DRIL\), which converts disagreement among cloned policies into a reinforcement\-learning reward, improves image\-based continuous control\.Methods:A controlled CarRacing study combines Gaussian and Beta learner policies, demonstrations from either a clipped Gaussian expert or an intrinsically bounded Beta expert, one or 20 trajectories, deterministic and stochastic evaluation, and three retained stages: behavior cloning, the highest 10\-episode training\-score checkpoint, and the final DRIL checkpoint\. The disagreement ensemble contains five Gaussian policies in every variant\. Each retained policy is evaluated over 100 procedurally generated episodes\.Results:Score\-selected DRIL produced its largest gains in the few\-demonstration setting, improving over the strongest behavior\-cloning mean by 61% with clipped\-action demonstrations and by 112% with bounded\-action demonstrations\. With 20 trajectories, the advantage of DRIL narrowed; in the bounded\-action regime, Beta behavior cloning remained about 7% above the best DRIL checkpoint\. The experiments also show that the informativeness of the disagreement reward changes with the learner representation and training stage\.Conclusion:DRIL can substantially improve few\-demonstration visual continuous control, while bounded Beta policies provide strong behavior\-cloning performance when more demonstrations are available\. The results highlight the joint importance of learner support, ensemble response, and checkpoint selection\.

###### keywords

imitation learning, behavior cloning, ensemble disagreement, autonomous driving, continuous control, Beta distribution

## 1Introduction

Learning control directly from demonstrations is a long\-standing approach to autonomous driving\. Early work mapped camera observations to steering commands with a neural network[Pomerleau \(1988\)](https://arxiv.org/html/2609.38407#bib.bib7), and later end\-to\-end systems scaled this idea with deep visual encoders and larger datasets[Bojarski et al\. \(2016\)](https://arxiv.org/html/2609.38407#bib.bib10);[Codevilla et al\. \(2018\)](https://arxiv.org/html/2609.38407#bib.bib11)\. The simplest form, behavior cloning \(BC\), treats expert state–action pairs as supervised examples\. BC is inexpensive compared with online reinforcement learning \(RL\), does not require a manually designed task reward, and can reuse logged demonstrations\. Its principal weakness is distribution shift: training minimizes error under the expert’s state distribution, while deployment generates states under the learned policy\. Small errors therefore compound and can lead to states that were never demonstrated[Ross and Bagnell \(2010\)](https://arxiv.org/html/2609.38407#bib.bib8);[Ross et al\. \(2011\)](https://arxiv.org/html/2609.38407#bib.bib9)\.

Interactive imitation\-learning methods reduce this mismatch by obtaining supervision on states visited by the learner\. DAgger aggregates labels from an expert during policy rollouts[Ross et al\. \(2011\)](https://arxiv.org/html/2609.38407#bib.bib9); active\-learning variants query a teacher when the policy is uncertain[Hussein et al\. \(2018\)](https://arxiv.org/html/2609.38407#bib.bib20); and hybrid RL–IL methods use environment interaction to refine a cloned policy[Kumar \(2024\)](https://arxiv.org/html/2609.38407#bib.bib22);[Lu et al\. \(2023\)](https://arxiv.org/html/2609.38407#bib.bib31)\. These approaches can improve recovery, but they may require repeated expert access, a task reward, or a learned reward model\.

Disagreement\-Regularized Imitation Learning \(DRIL\) offers a different mechanism[Brantley et al\. \(2020\)](https://arxiv.org/html/2609.38407#bib.bib12)\. An ensemble of policies is trained by BC on bootstrapped expert data\. Agreement among the ensemble members is expected on demonstrated states, whereas disagreement is treated as evidence that a rollout has left the demonstrated distribution\. The ensemble variance is thresholded to form a binary reward for PPO, and BC updates are interleaved with policy\-gradient updates\. DRIL therefore replaces explicit task\-reward design with a reward induced by imitation uncertainty\.

The original DRIL evaluation considered image observations with discrete actions \(Atari\) and low\-dimensional observations with continuous actions\. End\-to\-end autonomous control combines both challenges: high\-dimensional visual input and continuous bounded actions\. This combination raises two questions that are not independent\. First, does ensemble disagreement remain useful when the learner’s action is sampled from a continuous distribution? Second, how does the support of that distribution affect both BC and the disagreement signal?

Gaussian policies are common in continuous\-control RL, but their infinite support requires clipping when actuators are bounded\. The clipping operation changes the executed action and can bias policy\-gradient estimation[Chou et al\. \(2017\)](https://arxiv.org/html/2609.38407#bib.bib15)\. A Beta policy has finite support and generates valid actions by construction\. Previous work showed that replacing the Gaussian distribution with the Beta distribution improves PPO in CarRacing[Petrazzini and Antonelo \(2021\)](https://arxiv.org/html/2609.38407#bib.bib16)\. In imitation learning, however, a stronger bounded policy can change the state visitation distribution and the scale of ensemble variance\. It can therefore improve BC while simultaneously weakening the reward on which DRIL depends\.

The objective of this study is to determine whether disagreement regularization can improve behavior cloning in image\-based continuous control, and to analyze how learner\-policy support, expert\-action bounds, demonstration count, evaluation mode, and checkpoint selection affect the result\. The work advances the original DRIL evaluation by studying the previously untested combination of high\-dimensional visual observations and continuous bounded actions, while systematically analyzing the interaction among policy representation, demonstration source, dataset size, and checkpoint selection\.

The main contributions are:

1. 1\.an adaptation and evaluation of DRIL for stacked image observations and two\-dimensional continuous bounded actions;
2. 2\.a factorial comparison of Gaussian and Beta learner policies using demonstrations generated by either a clipped Gaussian expert or an intrinsically bounded Beta expert, with one and 20 trajectories;
3. 3\.a systematic comparison of BC, score\-selected transient DRIL checkpoints, and final DRIL checkpoints, exposing the dependence of the reported performance on model\-selection protocol;
4. 4\.an empirical analysis of training\-stage dynamics and of the disagreement\-reward response when a Beta learner is paired with the fixed Gaussian ensemble;
5. 5\.complete deterministic and stochastic 100\-episode evaluations for every retained policy configuration\.

The strongest result is not that one policy distribution dominates universally\. Score\-selected DRIL obtains the highest stochastic mean in three of four demonstration regimes, whereas Beta BC is strongest when 20 bounded\-action trajectories are available as a training set\. The evidence instead shows that the usefulness of disagreement regularization depends on the policy representation, the informativeness of the thresholded uncertainty reward, and the checkpoint that is retained\.

## 2Related work

### 2\.1Imitation learning and covariate shift

BC learns a direct mapping from observation to action and is widely used because it reduces imitation to standard supervised learning\. Its closed\-loop weakness motivated reductions from imitation learning to online learning and dataset aggregation[Ross and Bagnell \(2010\)](https://arxiv.org/html/2609.38407#bib.bib8);[Ross et al\. \(2011\)](https://arxiv.org/html/2609.38407#bib.bib9)\. More recent theoretical analysis has revisited the commonly assumed performance gap between offline behavior cloning and online imitation learning, showing that the horizon dependence of BC can be substantially improved under appropriate assumptions[Foster et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib2)\. Causal confusion is a related problem: a policy can exploit correlations in the demonstrations that do not remain valid after intervention or deployment[de Haan et al\. \(2019\)](https://arxiv.org/html/2609.38407#bib.bib33)\. Adversarial imitation methods such as GAIL instead match expert occupancy measures through a discriminator, but introduce adversarial optimization and usually require substantial environment interaction[Ho and Ermon \(2016\)](https://arxiv.org/html/2609.38407#bib.bib13)\. BC\-augmented GAIL methods attempt to improve sample efficiency and stability[Jena et al\. \(2021\)](https://arxiv.org/html/2609.38407#bib.bib14);[Couto and Antonelo \(2025\)](https://arxiv.org/html/2609.38407#bib.bib5)\.

Hussein et al\. used visual demonstrations for 3D navigation and refined the cloned policy through active learning on uncertain states[Hussein et al\. \(2018\)](https://arxiv.org/html/2609.38407#bib.bib20)\. Celemin and Kober explicitly separated epistemic uncertainty from demonstration ambiguity when deciding when corrective or evaluative human feedback was needed[Celemin and Kober \(2023\)](https://arxiv.org/html/2609.38407#bib.bib21)\. Kumar combined imitation guidance and RL for navigation in dynamic environments[Kumar \(2024\)](https://arxiv.org/html/2609.38407#bib.bib22)\. Chen and Xuan used adversarial inverse RL to infer a reward for visual continuous drone navigation[Chen and Xuan \(2026\)](https://arxiv.org/html/2609.38407#bib.bib24)\. These studies share the objective of moving beyond static cloning, but differ in the source of corrective information\. DRIL neither requests a new expert label nor attempts to recover the expert’s latent task reward; it constructs a surrogate reward from disagreement among cloned policies\.

The distribution\-shift problem also connects to offline RL, where a learned policy may select actions poorly represented in a fixed dataset\. A recent survey organizes solutions around policy constraints, uncertainty penalties, and out\-of\-distribution action control[Riahi Samani et al\. \(2026\)](https://arxiv.org/html/2609.38407#bib.bib23)\. DRIL is not an offline\-RL algorithm because it interacts with the environment during PPO training, but its binary disagreement reward acts as a constraint that encourages the policy to remain near the demonstrated distribution\.

Table 1:Qualitative comparison of methods that address imitation\-learning distribution shift\. “Expert queries” indicates whether additional labels are required after the initial demonstrations\. “Task reward” indicates use of the environment reward during policy improvement\.MethodExpert queriesTask rewardCorrective mechanismBCNoNoNone; supervised fitting on the fixed datasetDAgger[Ross et al\. \(2011\)](https://arxiv.org/html/2609.38407#bib.bib9)YesNoAggregates expert labels on learner\-visited statesGAIL[Ho and Ermon \(2016\)](https://arxiv.org/html/2609.38407#bib.bib13)NoNoAdversarial occupancy\-measure matchingHybrid RL–IL[Kumar \(2024\)](https://arxiv.org/html/2609.38407#bib.bib22);[Lu et al\. \(2023\)](https://arxiv.org/html/2609.38407#bib.bib31)VariesYesRefines imitation with environment rewardsDiffusion Meets DAgger[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib29)No new labelsNoSynthesizes examples near failure statesDRIL[Brantley et al\. \(2020\)](https://arxiv.org/html/2609.38407#bib.bib12)NoNoThresholded disagreement among cloned policies
### 2\.2Visual imitation learning for autonomous control

Autonomous\-driving imitation learning has evolved from direct camera\-to\-control mappings[Pomerleau \(1988\)](https://arxiv.org/html/2609.38407#bib.bib7);[Bojarski et al\. \(2016\)](https://arxiv.org/html/2609.38407#bib.bib10)to conditional policies and privileged teachers[Codevilla et al\. \(2018\)](https://arxiv.org/html/2609.38407#bib.bib11);[Zhang et al\. \(2021\)](https://arxiv.org/html/2609.38407#bib.bib32)\. Behavior cloning with Bird’s\-Eye View representations has also been investigated under limited demonstration data in simulated urban driving[Antonelo et al\. \(2025a\)](https://arxiv.org/html/2609.38407#bib.bib3)\. More recently, hybrid learning\-based controllers have combined neural control with model\-based expertise and online imitation learning in CARLA, providing another approach to adapting autonomous\-driving policies from expert demonstrations[Athni Hiremath and Bajcinca \(2026\)](https://arxiv.org/html/2609.38407#bib.bib6)\. Other work combines imitation and RL to improve robustness in difficult driving scenarios[Lu et al\. \(2023\)](https://arxiv.org/html/2609.38407#bib.bib31)\. Closed\-loop evaluation is essential because low supervised error does not guarantee route completion or recovery from deviations\. Modern benchmarks such as Bench2Drive evaluate multiple closed\-loop abilities and scenario types in CARLA[Dosovitskiy et al\. \(2017\)](https://arxiv.org/html/2609.38407#bib.bib36);[Jia et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib30)\. CarRacing remains useful for controlled experiments because tracks are generated procedurally, policy execution is closed loop, observations are images, and actions are continuous\.

### 2\.3Uncertainty and ensemble disagreement

Ensembles provide a practical estimate of epistemic uncertainty\. In DRIL, multiple BC policies are trained from different initializations and bootstrapped data\. Their prediction variance is expected to be small on demonstrated samples and larger elsewhere[Brantley et al\. \(2020\)](https://arxiv.org/html/2609.38407#bib.bib12)\. A high quantile of uncertainty on the expert data defines a threshold; crossing it produces a negative reward\. Empirical follow\-up studies have shown that deep imitation methods, including DRIL, are sensitive to implementation choices and to the interaction between BC and RL updates[Sasaki and Yamashina \(2021\)](https://arxiv.org/html/2609.38407#bib.bib34);[Arulkumaran and Ogawa Lillrank \(2024\)](https://arxiv.org/html/2609.38407#bib.bib35)\.

Recent work on runtime monitoring of generative imitation policies further shows that learned policies can exhibit qualitatively different failure modes, motivating complementary indicators beyond a single uncertainty or consistency signal[Agia et al\. \(2025\)](https://arxiv.org/html/2609.38407#bib.bib1)\.

The behavior of ensemble variance depends on both the policy representation and the states visited during rollout\. Celemin and Kober’s distinction between epistemic and aleatoric uncertainty[Celemin and Kober \(2023\)](https://arxiv.org/html/2609.38407#bib.bib21)is therefore directly relevant\. The experiments below analyze this interaction explicitly: for Beta learners, the fixed Gaussian ensemble often remains below its expert\-data variance threshold, producing an almost uniformly positive reward and revealing how the reward signal changes across learner representations\.

### 2\.4Continuous policy representations

A Gaussian policy is convenient for policy gradients but has unbounded support\. Clipping a sampled action to a bounded interval changes the action applied by the environment and can introduce gradient bias[Chou et al\. \(2017\)](https://arxiv.org/html/2609.38407#bib.bib15)\. The Beta distribution has finite support and avoids invalid samples\. Petrazzini and Antonelo previously compared Gaussian and Beta PPO policies in LunarLanderContinuous and CarRacing, finding better stability and performance for the Beta parameterization[Petrazzini and Antonelo \(2021\)](https://arxiv.org/html/2609.38407#bib.bib16)\. That prior PPO contribution is used here only to provide a bounded expert and motivate the learner parameterization; it is not claimed as a new contribution of this paper\.

Recent imitation\-learning policies are considerably more expressive than a unimodal Gaussian or Beta actor\. Implicit Behavioral Cloning represents the policy with an energy function[Florence et al\. \(2022\)](https://arxiv.org/html/2609.38407#bib.bib25); recent work has also explored energy\-based implicit behavior cloning for multimodal vehicle navigation in simulated cities[Antonelo et al\. \(2025b\)](https://arxiv.org/html/2609.38407#bib.bib4); Behavior Transformers model multimodal action distributions through discrete action clusters and residuals[Shafiullah et al\. \(2022\)](https://arxiv.org/html/2609.38407#bib.bib26); Diffusion Policy generates action sequences through conditional denoising[Chi et al\. \(2023\)](https://arxiv.org/html/2609.38407#bib.bib27); and VQ\-BeT uses latent action tokens for multimodal behavior generation[Lee et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib28)\. Diffusion Meets DAgger addresses covariate shift by synthesizing examples near failure states rather than querying a teacher[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib29)\. These methods provide a modern context for the explicit Gaussian and Beta policy representations evaluated in this study\.

## 3Method

### 3\.1Problem formulation and behavior cloning

Let𝒟=\{\(si,ai\)\}i=1N\\mathcal\{D\}=\\\{\(s\_\{i\},a\_\{i\}\)\\\}\_\{i=1\}^\{N\}be a set of expert state–action pairs\. A policyπθ​\(a∣s\)\\pi\_\{\\theta\}\(a\\mid s\)is trained to reproduce the expert\. For continuous actions, BC minimizes the mean\-squared error between the expert action and the policy mean,

ℒBC​\(θ\)=1N​∑i=1N‖μθ​\(si\)−ai‖22\.\\mathcal\{L\}\_\{\\mathrm\{BC\}\}\(\\theta\)=\\frac\{1\}\{N\}\\sum\_\{i=1\}^\{N\}\\left\\\|\\mu\_\{\\theta\}\(s\_\{i\}\)\-a\_\{i\}\\right\\\|\_\{2\}^\{2\}\.\(1\)The data are split randomly into 80% training and 20% validation samples\. Training stops after 20 epochs without improvement in validation loss, up to a maximum of 2000 epochs, and the best validation checkpoint initializes DRIL\.

For CarRacing, the action is two dimensional\. The first component is steering in\[−1,1\]\[\-1,1\]\. Throttle and brake are merged into a second component in\[−1,1\]\[\-1,1\], where−1\-1is full brake,\+1\+1is full throttle, and zero applies neither\. The merged action avoids simultaneous acceleration and braking and matches the policy used to generate the expert demonstrations\.

### 3\.2Disagreement ensemble

An ensembleΠE=\{πϕj\}j=1E\\Pi\_\{E\}=\\\{\\pi\_\{\\phi\_\{j\}\}\\\}\_\{j=1\}^\{E\}withE=5E=5is trained by BC\. Each member starts from a different random initialization and receives a bootstrap sample drawn with replacement, with the same size as the original demonstration dataset\. Ensemble members use the Gaussian policy architecture and minimize mean\-squared error between their predicted action means and the expert actions\. They are trained for exactly 2000 epochs without early stopping so that they fit the training set while retaining different extrapolations away from it\. This Gaussian ensemble is held fixed in design for all main experiments: in the DRIL\-Beta variants, only the learner actor is changed to a Beta policy\.

For a continuous action, each ensemble member predicts a mean vectormj​\(s\)∈ℝdm\_\{j\}\(s\)\\in\\mathbb\{R\}^\{d\}\. The empirical mean and covariance are

m¯​\(s\)\\displaystyle\\bar\{m\}\(s\)=1E​∑j=1Emj​\(s\),\\displaystyle=\\frac\{1\}\{E\}\\sum\_\{j=1\}^\{E\}m\_\{j\}\(s\),\(2\)ΣE​\(s\)\\displaystyle\\Sigma\_\{E\}\(s\)=1E​∑j=1E\(mj​\(s\)−m¯​\(s\)\)​\(mj​\(s\)−m¯​\(s\)\)⊤\.\\displaystyle=\\frac\{1\}\{E\}\\sum\_\{j=1\}^\{E\}\\big\(m\_\{j\}\(s\)\-\\bar\{m\}\(s\)\\big\)\\big\(m\_\{j\}\(s\)\-\\bar\{m\}\(s\)\\big\)^\{\\top\}\.\(3\)
Following the continuous\-action implementation used in this work, the covariance matrix is mapped to an action\-dependent scalar through the quadratic form

u⁡\(s,a\)=a⊤​ΣE​\(s\)​a\.u\(s,a\)=a^\{\\top\}\\Sigma\_\{E\}\(s\)a\.\(4\)
Equation \([4](https://arxiv.org/html/2609.38407#S3.E4)\) is the variance obtained after projecting the ensemble predictions onto the axis defined byaa, becauseVar⁡\(a⊤​M\)=a⊤​ΣE​\(s\)​a\\operatorname\{Var\}\(a^\{\\top\}M\)=a^\{\\top\}\\Sigma\_\{E\}\(s\)a\.

For the two\-dimensional actiona=\[asteer,along\]⊤a=\[a\_\{\\mathrm\{steer\}\},a\_\{\\mathrm\{long\}\}\]^\{\\top\}, this becomes

u⁡\(s,a\)=asteer2​Σ11\+2​asteer​along​Σ12\+along2​Σ22\.u\(s,a\)=a\_\{\\mathrm\{steer\}\}^\{2\}\\Sigma\_\{11\}\+2a\_\{\\mathrm\{steer\}\}a\_\{\\mathrm\{long\}\}\\Sigma\_\{12\}\+a\_\{\\mathrm\{long\}\}^\{2\}\\Sigma\_\{22\}\.\(5\)
Thus, only disagreement aligned withaacontributes to the scalar uncertainty measure, including the cross\-covariance between steering and longitudinal control\. Since the implementation uses the raw action vector rather than a unit vector, the score depends on both the action direction and its magnitude\.

The 98th percentile ofu⁡\(s,a\)u\(s,a\)over expert training samples definesq0\.98q\_\{0\.98\}\. The clipped uncertainty cost and equivalent reward are

CUclip​\(s,a\)\\displaystyle C\_\{U\}^\{\\mathrm\{clip\}\}\(s,a\)=\{−1,u⁡\(s,a\)≤q0\.98,\+1,u⁡\(s,a\)\>q0\.98,\\displaystyle=\\begin\{cases\}\-1,&u\(s,a\)\\leq q\_\{0\.98\},\\\\ \+1,&u\(s,a\)\>q\_\{0\.98\},\\end\{cases\}\(6\)rU​\(s,a\)\\displaystyle r\_\{U\}\(s,a\)=−CUclip​\(s,a\)\.\\displaystyle=\-C\_\{U\}^\{\\mathrm\{clip\}\}\(s,a\)\.\(7\)Thus, ensemble agreement yields\+1\+1and disagreement yields−1\-1, as reward signals\.

### 3\.3DRIL optimization

The BC\-initialized learner interacts with the environment using the uncertainty reward in Eq\. \([7](https://arxiv.org/html/2609.38407#S3.E7)\)\. PPO optimizes the clipped surrogate objective[Schulman et al\. \(2017\)](https://arxiv.org/html/2609.38407#bib.bib17), with advantages estimated by generalized advantage estimation[Schulman et al\. \(2015\)](https://arxiv.org/html/2609.38407#bib.bib18)\. One BC update is followed by one PPO policy\-gradient update for each optimization minibatch, preserving direct imitation while encouraging the rollout distribution to remain in low\-disagreement regions\. Figure[1](https://arxiv.org/html/2609.38407#S3.F1)summarizes the procedure\.

![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/fig1_dril_overview.png)Figure 1:Two\-stage DRIL workflow used in this work\. In stage \(a\), the expert demonstration dataset𝒟\\mathcal\{D\}trains the ensembleΠ\\Piby behavior cloning; in the present experiments,Π\\Picontains five Gaussian policies trained on bootstrap samples\. In stage \(b\), a BC\-initialized learner interacts with the environment\. Ensemble disagreement provides the binary reward for reinforcement\-learning updates, while behavior\-cloning updates continue to use the original demonstrations\.Algorithm 1DRIL training and checkpoint reportingInput:Expert data

𝒟\\mathcal\{D\}; ensemble size

E=5E=5; uncertainty quantile

0\.980\.98
1Train learner

π\\piby BC with an 80/20 split and retain the best validation checkpoint

πBC\\pi\_\{\\mathrm\{BC\}\};

2Set

π←πBC\\pi\\leftarrow\\pi\_\{\\mathrm\{BC\}\};

3for*e←1e\\leftarrow 1toEE*do

4Draw

𝒟e\\mathcal\{D\}\_\{e\}from

𝒟\\mathcal\{D\}with replacement, with

\|𝒟e\|=\|𝒟\|\|\\mathcal\{D\}\_\{e\}\|=\|\\mathcal\{D\}\|;

5Train Gaussian ensemble member

πe\\pi\_\{e\}on

𝒟e\\mathcal\{D\}\_\{e\}for 2000 epochs;

6Compute

q0\.98q\_\{0\.98\}from the ensemble uncertainty on expert training samples;

7Initialize score\-selected checkpoint

πpeak←πBC\\pi\_\{\\mathrm\{peak\}\}\\leftarrow\\pi\_\{\\mathrm\{BC\}\}and best 10\-episode mean

M←−∞M\\leftarrow\-\\infty;

8while*the DRIL interaction budget is not exhausted*do

9Collect a 2048\-step rollout with

π\\piusing the binary uncertainty reward;

10foreach*optimization minibatch*do

11Perform one gradient update to minimize

JBC​\(π\)J\_\{\\mathrm\{BC\}\}\(\\pi\)using a minibatch from

𝒟\\mathcal\{D\};

12Perform one PPO policy\-gradient update using the disagreement rewards from the rollout;

13if*the latest 10\-episode mean task score is at leastMM*then

14

πpeak←π\\pi\_\{\\mathrm\{peak\}\}\\leftarrow\\pi; update

MM;

15Set

πfinal←π\\pi\_\{\\mathrm\{final\}\}\\leftarrow\\pi;

Output:Behavior\-cloning policy

πBC\\pi\_\{\\mathrm\{BC\}\}; task\-score\-selected DRIL checkpoint

πpeak\\pi\_\{\\mathrm\{peak\}\}; final DRIL policy

πfinal\\pi\_\{\\mathrm\{final\}\}

The algorithm returns policies from three stages of the same learner\. The behavior\-cloning checkpointπBC\\pi\_\{\\mathrm\{BC\}\}is retained before any disagreement\-regularized policy\-gradient update and serves as the imitation baseline\. During DRIL training,πpeak\\pi\_\{\\mathrm\{peak\}\}is replaced whenever the moving mean of the latest 10 CarRacing task scores exceeds its previous maximum; it therefore represents the strongest transient checkpoint according to the environment score rather than the ensemble reward\. The final policyπfinal\\pi\_\{\\mathrm\{final\}\}is the learner obtained after the complete DRIL interaction budget, irrespective of whether an earlier checkpoint achieved a higher score\. Reporting all three policies separates the initial cloned solution, the best task\-score\-selected intermediate policy, and the policy produced at the prescribed end of training\.

The key distinction in this paper is between two retained DRIL checkpoints:

- •DRIL\-final: the policy after approximately 3 million environment steps \(about 3000 episodes\);
- •DRIL\-peak: the policy stored whenever the moving average of the previous 10 environment task scores exceeds its earlier maximum\.

DRIL\-peak uses the true CarRacing task score for checkpoint selection and is reported alongside both the BC initialization and the final DRIL checkpoint so that the complete training trajectory can be compared\.

### 3\.4Gaussian and Beta learner policies

For both Gaussian and Beta policies, the visual encoder comprises three convolutional layers followed by a 512\-unit fully connected layer \(Figure[2](https://arxiv.org/html/2609.38407#S3.F2)\)\. The actor has two additional 512\-unit fully connected layers, whereas the critic uses one separate 512\-unit fully connected layer followed by a scalar output\.

The Gaussian actor predicts the mean for steering and brake/throttle and uses a diagonal Gaussian distribution\. Sampled actions outside\[−1,1\]2\[\-1,1\]^\{2\}are clipped before execution\. In BC, Eq\. \([1](https://arxiv.org/html/2609.38407#S3.E1)\) directly constrains the mean, while the distribution scale is not identified by the supervised target\.

The Beta actor predicts four positive outputs,\(α1,β1,α2,β2\)\(\\alpha\_\{1\},\\beta\_\{1\},\\alpha\_\{2\},\\beta\_\{2\}\), using a softplus transformation and parameters greater than one\. Each Beta samplex∈\[0,1\]x\\in\[0,1\]is mapped toa=2​x−1a=2x\-1\. Its deterministic mean is

μBeta​\(s\)=2​α⁡\(s\)α⁡\(s\)\+β⁡\(s\)−1\.\\mu\_\{\\mathrm\{Beta\}\}\(s\)=2\\frac\{\\alpha\(s\)\}\{\\alpha\(s\)\+\\beta\(s\)\}\-1\.\(8\)Optimizing the mean changes both shape parameters and therefore also changes the variance\.

![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/fig3a_gaussian_architecture.jpg)\(a\)Gaussian learner: the actor outputs the means of two diagonal Gaussian action distributions for steering and brake/throttle\.
![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/fig3b_beta_architecture.jpg)\(b\)Beta learner: the actor outputsα\\alphaandβ\\betafor two independent bounded action distributions\.

Figure 2:Visual actor–critic architectures used in this work\. Both learners use the same image preprocessing and convolutional feature extractor\. The actor head changes from Gaussian outputs in \(a\) to Beta\-distribution parameters in \(b\), while the critic estimates a scalar value function\. The disagreement ensemble remains Gaussian in all experiments\.

## 4Experimental design

### 4\.1Staged implementation checks

Before the CarRacing study, the implementation was checked in the two regimes originally emphasized by DRIL\. Breakout combines image observations and discrete actions\. LunarLanderContinuous combines a low\-dimensional state and continuous actions\. These experiments are not treated as new algorithmic contributions; they verify that the implementation can reproduce the expected qualitative behavior before combining visual input and continuous actions\. Their numerical summary and corresponding figures are provided in Appendix[A](https://arxiv.org/html/2609.38407#A1)\.

### 4\.2CarRacing task and preprocessing

CarRacing\-v0 generates a new top\-down racing track for each episode[Brockman et al\. \(2016\)](https://arxiv.org/html/2609.38407#bib.bib19)\. The original observation is a96×9696\\times 96RGB image\. Each frame is converted to grayscale, resized to84×8484\\times 84, and stacked with the previous three frames, producing a4×84×844\\times 84\\times 84input\. The stack supplies short\-term motion information without an explicit speed sensor\.

The track is divided into tiles\. The agent receives1000/Ntiles1000/N\_\{\\mathrm\{tiles\}\}for each newly visited tile and loses 0\.1 per frame\. An episode ends after all tiles are visited, after 1000 frames, or when the vehicle moves too far from the track\. An average score above 900 over 100 consecutive episodes is the conventional solving criterion for this environment\. The 100\-episode evaluations use procedurally generated tracks that are distinct from the recorded demonstration trajectories, so they probe closed\-loop behavior on new track realizations\. For each retained policy, the reported mean and standard deviation are calculated over these 100 evaluation episodes\.

### 4\.3Clipped\-action and bounded\-action experts

Two expert sources produce the demonstration datasets:

1. 1\.a PPO policy with a Gaussian action distribution\. Approximately 73% of sampled expert actions fall outside the valid interval in the analyzed trajectory, so the executed and stored actions are clipped to\[−1,1\]2\[\-1,1\]^\{2\};
2. 2\.a PPO policy with a Beta action distribution\. Its support is mapped to\[−1,1\]2\[\-1,1\]^\{2\}, so every sampled action is bounded by construction\.

The selected Gaussian expert was reported at897±41897\\pm 41and the selected Beta expert at913±26913\\pm 26over 100 episodes\. The Beta PPO result and expert training were previously published[Petrazzini and Antonelo \(2021\)](https://arxiv.org/html/2609.38407#bib.bib16); they are used here as an expert\-data source\. For each source, 20 successful trajectories with episode score above 900 were collected\. Experiments use either the first trajectory or all 20 trajectories\.

Figure[3](https://arxiv.org/html/2609.38407#S4.F3)illustrates the different action distributions\. The clipped expert contains many saturated commands at the limits, whereas the bounded expert spreads actions within the valid interval\.

![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/fig2_expert_actions_cropped.jpg)Figure 3:Expert\-action generation\. The Beta expert produces bounded samples; the Gaussian expert produces many samples outside the action range; the clipped Gaussian actions are those executed in the environment and used in the clipped\-action demonstration dataset\.
### 4\.4Factorial comparison

The study crosses five experimental factors \(Table[2](https://arxiv.org/html/2609.38407#S4.T2)\)\. The resulting policies are evaluated in both deterministic mode \(distribution mean\) and stochastic mode \(sampling from the distribution\)\. The same definitions are used for BC, DRIL\-peak, and DRIL\-final\.

Table 2:Experimental factors in the main CarRacing study\. Every learner is evaluated in deterministic mode, using the distribution mean, and stochastic mode, sampling from the distribution\. BC denotes the validation\-selected behavior\-cloning checkpoint; DRIL\-peak is selected by the highest 10\-episode training\-score mean; DRIL\-final is the last checkpoint\.FactorLevelsExpert demonstration actionsClipped Gaussian; intrinsically bounded BetaNumber of expert trajectories1; 20Learner policy distributionGaussian; BetaRetained training stageBC; DRIL\-peak; DRIL\-finalEvaluation modeDeterministic mean; stochastic samplingThe principal PPO and BC settings are listed in Table[3](https://arxiv.org/html/2609.38407#S4.T3)\. Adam is used for optimization[Kingma and Ba \(2015\)](https://arxiv.org/html/2609.38407#bib.bib37)\. The experimental record documents one main training run for each CarRacing configuration\. Each retained policy is then evaluated over 100 consecutive procedurally generated tracks\. Consequently, the reported mean and standard deviation quantify episode\-to\-episode variability conditional on a trained checkpoint; they do not quantify variability across independent retraining seeds\.

Table 3:Main CarRacing training settings\. Ensemble members are Gaussian policies trained for 2000 epochs on bootstrap samples\. The learner is first trained by BC and then updated by interleaved PPO and BC steps during DRIL\.ParameterValueParameterValueBC learning rate2\.5×10−42\.5\\times 10^\{\-4\}BC mini\-batch32BC data split80/20BC patience20 epochsEnsemble size5Ensemble training2000 epochsUncertainty quantile0\.98PPO learning rate3×10−43\\times 10^\{\-4\}, annealedPPO rollout2048 stepsPPO epochs10PPO mini\-batches32Discountγ\\gamma0\.99GAEλ\\lambda0\.95Entropy coefficient0Value\-loss coefficient0\.5Max\. gradient norm0\.5DRIL horizon≈3×106\\approx 3\\times 10^\{6\}stepsPeak window10 episodesThe experimental comparison focuses on BC and DRIL because the objective is to isolate the effect of disagreement regularization under controlled changes in policy representation, demonstration source, and dataset size\. Several recent imitation\-learning methods discussed in Section 2, including energy\-based, transformer, and diffusion policies, target multimodal or temporally extended action generation and require substantially different policy architectures and training protocols\. Interactive approaches such as DAgger additionally require repeated expert labeling, while adversarial and hybrid RL–IL methods introduce different supervision or reward assumptions\. Direct numerical comparison would therefore confound the effect of disagreement regularization with changes in the learning paradigm\. Instead, BC provides the common supervised initialization and primary baseline for every experimental configuration, while the complete factorial evaluation isolates when DRIL improves upon that baseline\.

The implementation is additionally evaluated in Breakout and LunarLanderContinuous to verify its qualitative behavior in the two observation/action regimes originally considered by DRIL before evaluating their combination in CarRacing\.

## 5Results

Across Figs\.[4](https://arxiv.org/html/2609.38407#S5.F4)–[11](https://arxiv.org/html/2609.38407#S5.F11), points represent individual episode scores\. In the training plots, the orange curve is the 10\-episode moving average used for DRIL\-peak selection; in the evaluation plots, the orange horizontal line indicates the 100\-episode mean\.

### 5\.1Gaussian learner policies

Table[4](https://arxiv.org/html/2609.38407#S5.T4)reports all Gaussian learner results\. Under clipped\-action demonstrations, stochastic BC has a low mean with one trajectory \(30±\\pm67\), but improves to 437±\\pm115 with 20 trajectories\. Stochastic DRIL\-peak increases the mean to 322±\\pm208 and 802±\\pm197, respectively\. The final checkpoints fall to 39±\\pm79 and 218±\\pm108\. The same pattern appears under bounded\-action demonstrations: stochastic DRIL\-peak reaches 341±\\pm246 and 720±\\pm289, whereas DRIL\-final reaches only 184±\\pm130 and 240±\\pm129\.

Table 4:Gaussian learner evaluation on CarRacing\. Values are mean±\\pmstandard deviation over 100 evaluation episodes\. Bold values mark the highest mean scores within each evaluation mode and dataset regime\.ExpertMethod1 trajectory20 trajectoriesDeterministicStochasticDeterministicStochasticClippedBC125±\\pm11330±\\pm67171±\\pm124437±\\pm115DRIL\-peak166±\\pm131322±\\pm208423±\\pm211802±\\pm197DRIL\-final41±\\pm8039±\\pm79229±\\pm107218±\\pm108BoundedBC194±\\pm11375±\\pm47617±\\pm260137±\\pm70DRIL\-peak235±\\pm123341±\\pm246656±\\pm266720±\\pm289DRIL\-final202±\\pm128184±\\pm130246±\\pm120240±\\pm129For clipped\-action demonstrations, the DRIL training traces and stochastic evaluation distributions are shown in Figs\.[4](https://arxiv.org/html/2609.38407#S5.F4)and[5](https://arxiv.org/html/2609.38407#S5.F5), respectively\. The corresponding bounded\-action results are shown in Figs\.[6](https://arxiv.org/html/2609.38407#S5.F6)and[7](https://arxiv.org/html/2609.38407#S5.F7)\. In both demonstration regimes, the score rises early in DRIL training and later declines\. The 10\-episode moving\-average checkpoint therefore retains a substantially different policy from the final checkpoint\.

![Refer to caption](https://arxiv.org/html/2609.38407v1/gclip_1traj_training.png)\(a\)DRIL training with one demonstration\.
![Refer to caption](https://arxiv.org/html/2609.38407v1/gclip_20traj_training.png)\(b\)DRIL training with 20 demonstrations\.

Figure 4:Training dynamics of the Gaussian learner using clipped\-action expert demonstrations\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively\.\(a\)Stochastic evaluation with one demonstration\.\(b\)Stochastic evaluation with 20 demonstrations\.
Figure 5:Stochastic 100\-episode evaluation of the Gaussian learner using clipped\-action expert demonstrations\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively, and compare BC, DRIL\-peak, and DRIL\-final\. Deterministic results are reported in Table[4](https://arxiv.org/html/2609.38407#S5.T4)\.![Refer to caption](https://arxiv.org/html/2609.38407v1/gbound_1traj_training.png)\(a\)DRIL training with one demonstration\.
![Refer to caption](https://arxiv.org/html/2609.38407v1/gbound_20traj_training.png)\(b\)DRIL training with 20 demonstrations\.

Figure 6:Training dynamics of the Gaussian learner using bounded\-action expert demonstrations\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively\.\(a\)Stochastic evaluation with one demonstration\.\(b\)Stochastic evaluation with 20 demonstrations\.
Figure 7:Stochastic 100\-episode evaluation of the Gaussian learner using bounded\-action expert demonstrations\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively, and compare BC, DRIL\-peak, and DRIL\-final\. Deterministic results are reported in Table[4](https://arxiv.org/html/2609.38407#S5.T4)\.The stochastic Gaussian peak policy is generally stronger than its deterministic counterpart, indicating that action sampling can help during evaluation\.

### 5\.2Beta learner policies

Table[5](https://arxiv.org/html/2609.38407#S5.T5)reports the Beta learner results\. BC is much more competitive in stochastic mode than the Gaussian BC policy\. With 20 trajectories, stochastic Beta BC reaches 758±\\pm237 for clipped demonstrations and 794±\\pm227 for bounded demonstrations\. The latter is the highest mean in the 20\-trajectory bounded\-expert regime\.

With one trajectory, stochastic DRIL\-peak improves Beta BC from 216±\\pm108 to 348±\\pm230 for clipped demonstrations and from 161±\\pm156 to 200±\\pm184 for bounded demonstrations\. With 20 trajectories, DRIL\-peak remains below Beta BC: it reaches 572±\\pm280 and 738±\\pm277, respectively\. All four DRIL\-final stochastic means are between 99 and 267, showing a consistent decrease from the strongest intermediate checkpoints across both learner classes\.

Table 5:Beta learner evaluation on CarRacing with the fixed Gaussian disagreement ensemble\. Values are mean±\\pmstandard deviation over 100 evaluation episodes\. Bold values mark the highest mean scores within each evaluation mode and dataset regime\.ExpertMethod1 trajectory20 trajectoriesDeterministicStochasticDeterministicStochasticClippedBC163±\\pm67216±\\pm108252±\\pm143758±\\pm237DRIL\-peak229±\\pm142348±\\pm230210±\\pm115572±\\pm280DRIL\-final122±\\pm116143±\\pm111265±\\pm165241±\\pm140BoundedBC147±\\pm105161±\\pm156567±\\pm239794±\\pm227DRIL\-peak195±\\pm88200±\\pm184398±\\pm225738±\\pm277DRIL\-final123±\\pm12599±\\pm121275±\\pm144267±\\pm153For the Beta learner, clipped\-action training and stochastic evaluation results are shown in Figs\.[8](https://arxiv.org/html/2609.38407#S5.F8)and[9](https://arxiv.org/html/2609.38407#S5.F9); bounded\-action results are shown in Figs\.[10](https://arxiv.org/html/2609.38407#S5.F10)and[11](https://arxiv.org/html/2609.38407#S5.F11)\. The Beta BC policy often starts DRIL from a high score, especially with 20 demonstrations\. During the subsequent PPO phase, performance decreases from the BC initialization\. In the 20\-trajectory bounded case, the highest retained DRIL checkpoint remains below BC, making BC the strongest method for that regime\.

\(a\)DRIL training with one demonstration\.\(b\)DRIL training with 20 demonstrations\.
Figure 8:Training dynamics of the Beta learner using clipped\-action expert demonstrations and the fixed Gaussian disagreement ensemble\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively\.\(a\)Stochastic evaluation with one demonstration\.\(b\)Stochastic evaluation with 20 demonstrations\.
Figure 9:Stochastic 100\-episode evaluation of the Beta learner using clipped\-action expert demonstrations and the fixed Gaussian disagreement ensemble\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively, and compare BC, DRIL\-peak, and DRIL\-final\. Deterministic results are reported in Table[5](https://arxiv.org/html/2609.38407#S5.T5)\.![Refer to caption](https://arxiv.org/html/2609.38407v1/bbound_1traj_training.png)\(a\)DRIL training with one demonstration\.
![Refer to caption](https://arxiv.org/html/2609.38407v1/bbound_20traj_training.png)\(b\)DRIL training with 20 demonstrations\.

Figure 10:Training dynamics of the Beta learner using bounded\-action expert demonstrations and the fixed Gaussian disagreement ensemble\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively\.\(a\)Stochastic evaluation with one demonstration\.\(b\)Stochastic evaluation with 20 demonstrations\.
Figure 11:Stochastic 100\-episode evaluation of the Beta learner using bounded\-action expert demonstrations and the fixed Gaussian disagreement ensemble\. Panels \(a\) and \(b\) correspond to one and 20 demonstrations, respectively, and compare BC, DRIL\-peak, and DRIL\-final\. Deterministic results are reported in Table[5](https://arxiv.org/html/2609.38407#S5.T5)\.
### 5\.3Cross\-factor comparison

Table[6](https://arxiv.org/html/2609.38407#S5.T6)compares the strongest stochastic BC and DRIL\-peak policy in each demonstration regime\. DRIL\-peak has the highest mean in three regimes\. Relative to the strongest BC mean, the increases are 61\.1% for one clipped trajectory, 5\.8% for 20 clipped trajectories, and 111\.8% for one bounded trajectory\. With 20 bounded trajectories, the best BC mean is 7\.1% higher than the best DRIL\-peak mean\. These percentages summarize the relative differences between the reported means\.

Table 6:Best stochastic BC and score\-selected DRIL checkpoint in each demonstration regime, allowing the learner distribution to differ\. Percent change is computed from the reported means\.ExpertTrajectoriesBest BCBest DRIL\-peakChange of DRIL peakClipped1Beta: 216±\\pm108Beta: 348±\\pm230\+61\.1%Clipped20Beta: 758±\\pm237Gaussian: 802±\\pm197\+5\.8%Bounded1Beta: 161±\\pm156Gaussian: 341±\\pm246\+111\.8%Bounded20Beta: 794±\\pm227Beta: 738±\\pm277−7\.1\-7\.1%Three interactions stand out\. First, stochastic evaluation is usually better than deterministic evaluation for the strongest checkpoints, particularly for Beta BC and Gaussian DRIL\-peak\. Second, the preferred learner distribution depends on the demonstration regime rather than solely on the expert policy family\. For example, with one bounded\-action trajectory generated by the Beta expert, the Gaussian learner achieves the strongest DRIL\-peak result, whereas with 20 bounded\-action trajectories the Beta learner provides the strongest overall result\. Third, more demonstrations reduce the headroom for DRIL when BC is already strong\. The 20\-trajectory bounded dataset produces a Beta BC policy with a high mean, and its mean remains above the retained DRIL checkpoints\.

### 5\.4Reward response with the Beta learner

Figures[12](https://arxiv.org/html/2609.38407#S5.F12)and[13](https://arxiv.org/html/2609.38407#S5.F13)contrast the first 1000 DRIL steps for Gaussian and Beta learners\. For the Gaussian learner, many sampled actions lie outside the valid range before clipping, and occasional ensemble variances exceed the threshold\. For the Beta learner, all actions are bounded\. More importantly, the ensemble variance often remains below the threshold even when the episode score indicates low task performance or departure from the track\. The resulting reward remains close to\+1\+1for most steps\.

![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/fig8a_gaussian_signal.jpg)Figure 12:Disagreement signal during the first 1000 interaction steps of the Gaussian learner\. The left column uses one expert trajectory and the right column uses 20 trajectories\. The top rows show steering and brake/throttle samples with the valid action interval shaded; the middle rows show scalar ensemble uncertainty and the 98th\-percentile threshold; the bottom rows show the resulting binary reward\. Sampled Gaussian actions frequently extend beyond the valid interval before clipping, and occasional uncertainty values cross the threshold\.![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/fig8b_beta_signal.jpg)Figure 13:Disagreement signal during the first 1000 interaction steps of the Beta learner using the same fixed Gaussian ensemble\. The left column uses one expert trajectory and the right column uses 20 trajectories\. The top rows show bounded steering and brake/throttle actions; the middle rows show scalar ensemble uncertainty and the 98th\-percentile threshold; the bottom rows show the binary reward\. Threshold crossings are infrequent, so the reward remains positive over most of the rollout\.A nearly constant reward gives PPO limited information for distinguishing actions during policy improvement, while PPO updates continue to move the actor away from the BC initialization\. This explains the observed decrease in Beta\-policy performance during DRIL training\. Bounded actions remain beneficial for BC, but they also change the interaction between the learner and the fixed uncertainty estimator\.

## 6Discussion

### 6\.1Checkpoint selection and training dynamics

The difference between DRIL\-peak and DRIL\-final shows that checkpoint selection is an important part of the experimental analysis\. For clipped demonstrations with 20 trajectories, the stochastic Gaussian mean falls from 802 at the selected peak to 218 at the final checkpoint\. For bounded demonstrations with 20 trajectories, it falls from 720 to 240\. Similar declines occur for Beta learners\. Reporting both checkpoints captures the transient improvement and the subsequent evolution of the policy, providing a more complete characterization of DRIL training\.

This evolution may reflect retention dynamics between the BC and PPO objectives\. Candidate mechanisms include the binary reward, the balance and ordering of BC and PPO updates, value\-function learning, the fixed uncertainty threshold, and changes in the learner distribution\. Additional experiments examined modifications to the disagreement reward, larger ensembles, and wider ensemble networks, but these changes did not eliminate the observed degradation\. The score\-selected checkpoint nevertheless provides a consistent way to identify the strongest intermediate policy in simulation\.

### 6\.2Policy support and ensemble response

The Beta policy has two clear advantages for BC\. Every sampled action lies within the actuator range, and optimizingα\\alphaandβ\\betachanges both the policy mean and concentration\. This likely explains why stochastic Beta BC is strong with 20 demonstrations\. The Gaussian BC objective constrains the mean but leaves the supervised objective with limited control over stochastic spread, which contributes to the lower one\-trajectory stochastic mean\.

The same boundedness changes DRIL’s operating conditions\. DRIL assumes that ensemble disagreement separates familiar from unfamiliar state–action pairs\. In the Beta experiments, bounded rollouts can obtain low task scores without creating large action\-direction variance in the fixed Gaussian ensemble\. This shows that keeping actions within the valid actuator range does not necessarily make the disagreement signal informative about whether the learner has moved outside the demonstrated distribution\. A policy can remain within actuator limits and still visit states that require a negative uncertainty reward\.

### 6\.3Expert demonstrations and dataset size

The clipped and bounded experts differ both in action support and in the policy that generated the trajectories\. Saturated actions in the clipped dataset can simplify some supervised mappings because many targets lie at−1\-1or\+1\+1, but they can also create abrupt control commands\. The bounded expert produces smoother interior actions and higher expert evaluation performance\. The two demonstration sources favor different learner/checkpoint combinations: the largest score, 802±\\pm197, comes from a Gaussian DRIL\-peak learner trained on 20 clipped trajectories, while the strongest pure BC result, 794±\\pm227, comes from a Beta learner trained on 20 bounded trajectories\.

Increasing the dataset from one to 20 trajectories generally improves BC, as expected\. DRIL provides its clearest relative benefit when one trajectory leaves large coverage gaps\. With 20 bounded demonstrations, BC is already strong and DRIL has little room to improve\. This pattern is consistent with the motivation for disagreement regularization: it is most useful when demonstration coverage is sparse and the initial cloned policy benefits from an additional online signal\.

### 6\.4Relation to current imitation\-learning methods

Modern generative policies can represent multimodal or temporally extended actions more effectively than the explicit Gaussian and Beta actors studied here[Florence et al\. \(2022\)](https://arxiv.org/html/2609.38407#bib.bib25);[Shafiullah et al\. \(2022\)](https://arxiv.org/html/2609.38407#bib.bib26);[Chi et al\. \(2023\)](https://arxiv.org/html/2609.38407#bib.bib27);[Lee et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib28)\. Covariate shift can still occur because a highly expressive policy may encounter out\-of\-distribution states\. Recent work therefore combines expressive policies with active data collection or synthetic augmentation[Zhang et al\. \(2024\)](https://arxiv.org/html/2609.38407#bib.bib29)\. The present findings remain relevant because any uncertainty\-driven correction mechanism must be calibrated to the chosen policy representation\. A richer multimodal policy can assign high probability to several distinct but valid actions for the same observation\. In that case, disagreement between policy outputs may reflect legitimate behavioral ambiguity rather than epistemic uncertainty, making action\-level disagreement harder to interpret as an out\-of\-distribution signal\.

## 7Conclusion

This paper examined disagreement\-regularized imitation learning in a regime that combines stacked visual observations with continuous bounded actions\. The study crossed learner distribution, expert\-action source, demonstration count, evaluation mode, and checkpoint choice\. Score\-selected DRIL checkpoints obtained the highest stochastic mean in three of four demonstration regimes, with the largest relative gains occurring in one\-trajectory settings\. When 20 bounded\-action demonstrations were available, stochastic Beta BC was the strongest method, showing that an intrinsically bounded policy can provide a competitive alternative to online disagreement regularization when demonstration coverage is broader\.

The results also clarify the interaction between policy support and the disagreement reward\. Gaussian and Beta learners induce different rollout distributions, and the fixed Gaussian ensemble responds differently to them\. For Beta learners, the reward often remains positive over long portions of the rollout, while the strongest intermediate checkpoints occur early in training\. Reporting BC, score\-selected, and final checkpoints therefore reveals complementary stages of the learning process and shows when DRIL adds value beyond the cloned initialization\.

Future work will extend the complete factorial study across independent training seeds and a shared set of evaluation\-track seeds, construct matched experts to separate action support from expert quality, and investigate policy\-matched ensembles for Beta learners\. Further directions include reward\-free checkpoint criteria based on held\-out demonstrations or learned success estimators, continuous or representation\-level disagreement signals, explicit counterexample states, and comparisons with DAgger, energy\-based, transformer, and diffusion policies\. Evaluation in scenario\-rich closed\-loop benchmarks such as CARLA and Bench2Drive will further test how the observed interactions transfer to urban traffic, route commands, weather variation, and safety\-critical events\. Overall, the study shows that DRIL can be especially effective in few\-demonstration visual continuous control when the ensemble reward remains informative throughout training\.

## Appendix AImplementation\-validation experiments

Before the CarRacing adaptation, the implementation was checked in the two observation/action regimes evaluated by the original DRIL study: visual observations with discrete actions in Breakout and low\-dimensional observations with continuous actions in LunarLanderContinuous\. These experiments verify the qualitative behavior of the implementation before combining visual observations and continuous actions in the main study\.

Table 7:Implementation checks in the two regimes considered by the original DRIL study\. Each entry is the mean score over 100 evaluation episodes\. The expert threshold is the task score used to define successful or expert\-level performance in the corresponding experiment\.EnvironmentDemonstrationsExpert thresholdBCDRILBreakout1 trajectory3005355Breakout3 trajectories3007339LunarLanderContinuous1 trajectory200194257LunarLanderContinuous20 trajectories200232228![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/figA1_breakout.jpg)\(a\)Breakout evaluation\. Each point is the score from one of 100 consecutive episodes\. The left panel uses one expert trajectory and the right panel uses three trajectories\. Horizontal lines show the DRIL mean, the BC mean, and the expert\-level threshold of 300\. DRIL reaches means of 355 and 339, compared with BC means of 5 and 7\.
![Refer to caption](https://arxiv.org/html/2609.38407v1/figures/figA2_lunarlander.jpg)\(b\)LunarLanderContinuous evaluation\. Each point is the score from one of 100 consecutive episodes\. The left panel uses one expert trajectory and the right panel uses 20 trajectories\. Horizontal lines show the DRIL mean, the BC mean, and the expert\-level threshold of 200\. DRIL changes the one\-trajectory mean from 194 to 257 and the 20\-trajectory mean from 232 to 228\.

Figure 14:Implementation\-validation results\. The figures show the qualitative behavior of DRIL in the original discrete\-visual and continuous\-low\-dimensional regimes before the image\-based continuous\-control study\.
## Statements and Declarations

Funding:This work was partially funded by the National Council for Scientific and Technological Development \- CNPq, Brazil \(Grant No\. 420148/2025\-6\) and in part by the Coordenação de Aperfeiçoamento de Pessoal de Nível Superior – Brasil \(CAPES\) – Finance Code 001\.

Competing interests:The authors declare that they have no competing interests\.

Ethics approval:Not applicable\. The study uses simulated control environments and does not involve human participants or animals\.

Author contributions:Irving Giovani Bronzatti Petrazzini contributed to methodology, software, investigation, validation, formal analysis, visualization, data curation, and writing–original draft\. Eric Aislan Antonelo contributed to conceptualization, methodology, supervision, project administration, formal analysis, and writing–review and editing\. Both authors read and approved the manuscript\.

## References

- C\. Agia, R\. Sinha, J\. Yang, Z\. Cao, R\. Antonova, M\. Pavone, and J\. BohgUnpacking failure modes of generative policies: runtime monitoring of consistency and progress\.InProceedings of The 8th Conference on Robot Learning,P\. Agrawal, O\. Kroemer, and W\. Burgard \(Eds\.\),Proceedings of Machine Learning Research, Vol\.270,pp\. 689–723\.External Links:[Link](https://proceedings.mlr.press/v270/agia25a.html)Cited by:[§2\.3](https://arxiv.org/html/2609.38407#S2.SS3.p2.1)\.
- Antoneloet al\.\(2025a\)E\. A\. Antonelo, G\. C\. K\. Couto, C\. Möller, and P\. H\. FernandesInvestigating behavior cloning from few demonstrations for autonomous driving based on bird’s\-eye view in simulated cities\.InIntelligent Systems,A\. Paes and F\. A\. N\. Verri \(Eds\.\),Cham,pp\. 155–168\.External Links:ISBN 978\-3\-031\-79032\-4Cited by:[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Antoneloet al\.\(2025b\)E\. Antonelo, G\. Couto, and C\. MöllerExploring multimodal implicit behavior learning for vehicle navigation in simulated cities\.InProceedings of the 22nd National Meeting on Artificial and Computational Intelligence,Porto Alegre, RS, Brasil,pp\. 962–973\.External Links:ISSN 2763\-9061,[Document](https://dx.doi.org/10.5753/eniac.2025.14289),[Link](https://sol.sbc.org.br/index.php/eniac/article/view/38783)Cited by:[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p2.1)\.
- Arulkumaran and Ogawa Lillrank \(2024\)K\. Arulkumaran and D\. Ogawa LillrankA pragmatic look at deep imitation learning\.InProceedings of the 15th Asian Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.222,pp\. 58–73\.Cited by:[§2\.3](https://arxiv.org/html/2609.38407#S2.SS3.p1.1)\.
- Athni Hiremath and Bajcinca \(2026\)S\. Athni Hiremath and N\. BajcincaA lifelong learnable hybrid controller for autonomous driving\.SN Computer Science7\(1\),pp\. 10\.External Links:[Document](https://dx.doi.org/10.1007/s42979-025-04635-5)Cited by:[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Bojarskiet al\.\(2016\)M\. Bojarski, D\. Del Testa, D\. Dworakowski, B\. Firner, B\. Flepp, P\. Goyal, L\. D\. Jackel, M\. Monfort, U\. Muller, J\. Zhang, X\. Zhang, J\. Zhao, and K\. ZiebaEnd to end learning for self\-driving cars\.arXiv preprint arXiv:1604\.07316\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1604.07316)Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Brantleyet al\.\(2020\)K\. Brantley, W\. Sun, and M\. HenaffDisagreement\-regularized imitation learning\.InInternational Conference on Learning Representations,Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p3.1),[§2\.3](https://arxiv.org/html/2609.38407#S2.SS3.p1.1),[Table 1](https://arxiv.org/html/2609.38407#S2.T1.5.7.1.1)\.
- Brockmanet al\.\(2016\)G\. Brockman, V\. Cheung, L\. Pettersson, J\. Schneider, J\. Schulman, J\. Tang, and W\. ZarembaOpenAI Gym\.arXiv preprint arXiv:1606\.01540\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1606.01540)Cited by:[§4\.2](https://arxiv.org/html/2609.38407#S4.SS2.p1.1)\.
- Celemin and Kober \(2023\)C\. Celemin and J\. KoberKnowledge\- and ambiguity\-aware robot learning from corrective and evaluative feedback\.Neural Computing and Applications35,pp\. 16821–16839\.External Links:[Document](https://dx.doi.org/10.1007/s00521-022-08118-z)Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p2.1),[§2\.3](https://arxiv.org/html/2609.38407#S2.SS3.p3.1)\.
- Chen and Xuan \(2026\)Z\. Chen and J\. XuanLearning unknown reward function for drone navigation based on inverse deep reinforcement learning\.Neural Computing and Applications38,pp\. 51\.External Links:[Document](https://dx.doi.org/10.1007/s00521-025-11698-1)Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p2.1)\.
- Chiet al\.\(2023\)C\. Chi, S\. Feng, Y\. Du, Z\. Xu, E\. Cousineau, B\. C\. M\. Burchfiel, and S\. SongDiffusion policy: visuomotor policy learning via action diffusion\.InProceedings of Robotics: Science and Systems,Daegu, Republic of Korea\.External Links:[Document](https://dx.doi.org/10.15607/RSS.2023.XIX.026)Cited by:[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p2.1),[§6\.4](https://arxiv.org/html/2609.38407#S6.SS4.p1.1)\.
- Chouet al\.\(2017\)P\. Chou, D\. Maturana, and S\. SchererImproving stochastic policy gradients in continuous control with deep reinforcement learning using the beta distribution\.InProceedings of the 34th International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.70,pp\. 834–843\.Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p5.1),[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p1.1)\.
- Codevillaet al\.\(2018\)F\. Codevilla, M\. Müller, A\. López, V\. Koltun, and A\. DosovitskiyEnd\-to\-end driving via conditional imitation learning\.In2018 IEEE International Conference on Robotics and Automation \(ICRA\),pp\. 4693–4700\.External Links:[Document](https://dx.doi.org/10.1109/ICRA.2018.8460487)Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Couto and Antonelo \(2025\)G\. C\. K\. Couto and E\. A\. AntoneloHierarchical generative adversarial imitation learning with mid\-level input generation for autonomous driving on urban environments\.IEEE Transactions on Intelligent Vehicles10\(3\),pp\. 1841–1854\.External Links:[Document](https://dx.doi.org/10.1109/TIV.2024.3436587)Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1)\.
- de Haanet al\.\(2019\)P\. de Haan, D\. Jayaraman, and S\. LevineCausal confusion in imitation learning\.InAdvances in Neural Information Processing Systems,Vol\.32,pp\. 11698–11709\.Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1)\.
- Dosovitskiyet al\.\(2017\)A\. Dosovitskiy, G\. Ros, F\. Codevilla, A\. López, and V\. KoltunCARLA: an open urban driving simulator\.InProceedings of the 1st Annual Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.78,pp\. 1–16\.Cited by:[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Florenceet al\.\(2022\)P\. Florence, C\. Lynch, A\. Zeng, O\. A\. Ramirez, A\. Wahid, L\. Downs, A\. Wong, J\. Lee, I\. Mordatch, and J\. TompsonImplicit behavioral cloning\.InProceedings of the 5th Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.164,pp\. 158–168\.Cited by:[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p2.1),[§6\.4](https://arxiv.org/html/2609.38407#S6.SS4.p1.1)\.
- Fosteret al\.\(2024\)D\. J\. Foster, A\. Block, and D\. MisraIs behavior cloning all you need? understanding horizon in imitation learning\.InAdvances in Neural Information Processing Systems,A\. Globerson, L\. Mackey, D\. Belgrave, A\. Fan, U\. Paquet, J\. Tomczak, and C\. Zhang \(Eds\.\),Vol\.37,pp\. 120602–120666\.External Links:[Document](https://dx.doi.org/10.52202/079017-3834),[Link](https://proceedings.neurips.cc/paper_files/paper/2024/file/da84e39ae51fd26bb5110d9659c06e13-Paper-Conference.pdf)Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1)\.
- Ho and Ermon \(2016\)J\. Ho and S\. ErmonGenerative adversarial imitation learning\.InAdvances in Neural Information Processing Systems,Vol\.29,pp\. 4565–4573\.Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2609.38407#S2.T1.5.4.1.1)\.
- Husseinet al\.\(2018\)A\. Hussein, E\. Elyan, M\. M\. Gaber, and C\. JayneDeep imitation learning for 3D navigation tasks\.Neural Computing and Applications29,pp\. 389–404\.External Links:[Document](https://dx.doi.org/10.1007/s00521-017-3241-z)Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p2.1)\.
- Jenaet al\.\(2021\)R\. Jena, C\. Liu, and K\. SycaraAugmenting GAIL with BC for sample efficient imitation learning\.InProceedings of the 2020 Conference on Robot Learning,Proceedings of Machine Learning Research, Vol\.155,pp\. 80–90\.Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1)\.
- Jiaet al\.\(2024\)X\. Jia, Z\. Yang, Q\. Li, Z\. Zhang, and J\. YanBench2Drive: towards multi\-ability benchmarking of closed\-loop end\-to\-end autonomous driving\.InAdvances in Neural Information Processing Systems,Vol\.37\.External Links:[Document](https://dx.doi.org/10.52202/079017-0025)Cited by:[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Kingma and Ba \(2015\)D\. P\. Kingma and J\. BaAdam: a method for stochastic optimization\.InInternational Conference on Learning Representations,Cited by:[§4\.4](https://arxiv.org/html/2609.38407#S4.SS4.p2.1)\.
- Kumar \(2024\)D\. KumarReinforcement imitation learning for reliable and efficient autonomous navigation in complex environments\.Neural Computing and Applications36,pp\. 11945–11961\.External Links:[Document](https://dx.doi.org/10.1007/s00521-024-09678-y)Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p2.1),[Table 1](https://arxiv.org/html/2609.38407#S2.T1.5.5.1.1)\.
- Leeet al\.\(2024\)S\. Lee, Y\. Wang, H\. Etukuru, H\. J\. Kim, N\. M\. M\. Shafiullah, and L\. PintoBehavior generation with latent actions\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 26991–27008\.Cited by:[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p2.1),[§6\.4](https://arxiv.org/html/2609.38407#S6.SS4.p1.1)\.
- Luet al\.\(2023\)Y\. Lu, J\. Fu, G\. Tucker, X\. Pan, E\. Bronstein, R\. Roelofs, B\. Sapp, B\. White, A\. Faust, S\. Whiteson, D\. Anguelov, and S\. LevineImitation is not enough: robustifying imitation with reinforcement learning for challenging driving scenarios\.In2023 IEEE/RSJ International Conference on Intelligent Robots and Systems \(IROS\),pp\. 7553–7560\.External Links:[Document](https://dx.doi.org/10.1109/IROS55552.2023.10342038)Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p2.1),[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1),[Table 1](https://arxiv.org/html/2609.38407#S2.T1.5.5.1.1)\.
- Petrazzini and Antonelo \(2021\)I\. G\. B\. Petrazzini and E\. A\. AntoneloProximal policy optimization with continuous bounded action space via the beta distribution\.In2021 IEEE Symposium Series on Computational Intelligence \(SSCI\),pp\. 1–8\.External Links:[Document](https://dx.doi.org/10.1109/SSCI50451.2021.9660123)Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p5.1),[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p1.1),[§4\.3](https://arxiv.org/html/2609.38407#S4.SS3.p1.2)\.
- Pomerleau \(1988\)D\. A\. PomerleauALVINN: an autonomous land vehicle in a neural network\.InAdvances in Neural Information Processing Systems,Vol\.1,pp\. 305–313\.Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.
- Riahi Samaniet al\.\(2026\)A\. Riahi Samani, X\. Zhao, and F\. ChenDistribution shift, generalization and OOD challenge in offline reinforcement learning: a comprehensive survey\.Neural Computing and Applications38,pp\. 150\.External Links:[Document](https://dx.doi.org/10.1007/s00521-026-11966-8)Cited by:[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p3.1)\.
- Ross and Bagnell \(2010\)S\. Ross and D\. BagnellEfficient reductions for imitation learning\.InProceedings of the Thirteenth International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.9,pp\. 661–668\.Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p1.1),[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1)\.
- Rosset al\.\(2011\)S\. Ross, G\. J\. Gordon, and J\. A\. BagnellA reduction of imitation learning and structured prediction to no\-regret online learning\.InProceedings of the Fourteenth International Conference on Artificial Intelligence and Statistics,Proceedings of Machine Learning Research, Vol\.15,pp\. 627–635\.Cited by:[§1](https://arxiv.org/html/2609.38407#S1.p1.1),[§1](https://arxiv.org/html/2609.38407#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.38407#S2.SS1.p1.1),[Table 1](https://arxiv.org/html/2609.38407#S2.T1.5.3.1.1)\.
- Sasaki and Yamashina \(2021\)F\. Sasaki and R\. YamashinaBehavioral cloning from noisy demonstrations\.InInternational Conference on Learning Representations,Cited by:[§2\.3](https://arxiv.org/html/2609.38407#S2.SS3.p1.1)\.
- Schulmanet al\.\(2015\)J\. Schulman, P\. Moritz, S\. Levine, M\. I\. Jordan, and P\. AbbeelHigh\-dimensional continuous control using generalized advantage estimation\.arXiv preprint arXiv:1506\.02438\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1506.02438)Cited by:[§3\.3](https://arxiv.org/html/2609.38407#S3.SS3.p1.1)\.
- Schulmanet al\.\(2017\)J\. Schulman, F\. Wolski, P\. Dhariwal, A\. Radford, and O\. KlimovProximal policy optimization algorithms\.arXiv preprint arXiv:1707\.06347\.External Links:[Document](https://dx.doi.org/10.48550/arXiv.1707.06347)Cited by:[§3\.3](https://arxiv.org/html/2609.38407#S3.SS3.p1.1)\.
- Shafiullahet al\.\(2022\)N\. M\. Shafiullah, Z\. Cui, A\. Altanzaya, and L\. PintoBehavior transformers: cloningkkmodes with one stone\.InAdvances in Neural Information Processing Systems,Vol\.35,pp\. 22955–22968\.External Links:[Document](https://dx.doi.org/10.52202/068431-1668)Cited by:[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p2.1),[§6\.4](https://arxiv.org/html/2609.38407#S6.SS4.p1.1)\.
- Zhanget al\.\(2024\)X\. Zhang, M\. Chang, P\. Kumar, and S\. GuptaDiffusion meets DAgger: supercharging eye\-in\-hand imitation learning\.InProceedings of Robotics: Science and Systems,Delft, The Netherlands\.External Links:[Document](https://dx.doi.org/10.15607/RSS.2024.XX.048)Cited by:[§2\.4](https://arxiv.org/html/2609.38407#S2.SS4.p2.1),[Table 1](https://arxiv.org/html/2609.38407#S2.T1.5.6.1.1),[§6\.4](https://arxiv.org/html/2609.38407#S6.SS4.p1.1)\.
- Zhanget al\.\(2021\)Z\. Zhang, A\. Liniger, D\. Dai, F\. Yu, and L\. Van GoolEnd\-to\-end urban driving by imitating a reinforcement learning coach\.InProceedings of the IEEE/CVF International Conference on Computer Vision \(ICCV\),pp\. 15222–15232\.Cited by:[§2\.2](https://arxiv.org/html/2609.38407#S2.SS2.p1.1)\.