A latent control model for realistic rodent motion

# A latent control model for realistic rodent motion
Source: [https://arxiv.org/html/2610.27279](https://arxiv.org/html/2610.27279)
Aidan Sirbu1,2,3,🖂\{\}^\{1,2,3,\\text\{\\Letter\}\}Charles Y\. Zhang4Yuanjia Yang6,7 Bence P\. Ölveczky5Talmo D\. Pereira6Blake A\. Richards1,2,3,8,9,10 1Google, Paradigms of Intelligence Team2Mila – Quebec AI Institute3School of Computer Science, McGill University4Harvard University5Department of Organismal and Evolutionary Biology, Harvard University6Salk Institute for Biological Studies7University of California San Diego8Department of Neurology and Neurosurgery, McGill University9Montreal Neurological Institute, McGill University10Learning in Machines and Brains Program, CIFAR🖂\{\}^\{\\text\{\\Letter\}\}[aidan\.sirbu@mail\.mcgill\.ca](mailto:[email protected])[https://aidansirbu\.com/SCAMPER\-site/](https://aidansirbu.com/SCAMPER-site/)

###### Abstract

The development of embodied models of animal behavior is a major goal of the neuro\-AI community\. Embodied models will enhance our ability to make direct comparisons between biological and artificial agents\. To make such comparisons effective, though, we need embodied models that actually move like animals\. Previous work has developed virtual animal bodies and techniques for training realistic motion through imitation\. However, to date, there is no system for embodied animal modeling that produces natural movements both during trained goal\-directed behavior, and during exploration driven by random generation\. Here, we introduceSimulatedControl viaAugmentedMotorPriors forEmbodiedReinforcement learning \(SCAMPER\), a system for training an embodied virtual rat model, enabling natural movements during both goal\-directed behavior and random exploration\. SCAMPER uses a variational architecture with two distinct priors, a motion prior for control and a behavior prior for exploration\. We show that combining both of these priors allows SCAMPER to generate realistic behavior during exploration and exploit the task structure effectively\. SCAMPER can also learn from sparse rewards as a result of the diverse movements induced by the behavior prior, allowing the system to model realistic sparse\-reward tasks from experimental neuroscience\. Altogether, SCAMPER provides the neuro\-AI community with an effective new system for training embodied models of rodents on realistic tasks\.

## 1Introduction

One of the ultimate goals of neuro\-AI research is to be able to fully capture the richness of animal learning and behavior with embodied computational models\[[1](https://arxiv.org/html/2610.27279#bib.bib1),[2](https://arxiv.org/html/2610.27279#bib.bib2)\]\. Previous work has developed realistic virtual bodies for a variety of model organisms commonly used in neuroscience experiments\[[3](https://arxiv.org/html/2610.27279#bib.bib3)\]\. However, controlling a realistic virtual body to consistently produce natural motion is non\-trivial\. For example, when an artificial neural network was trained to control a virtual rat on a variety of tasks using deep reinforcement learning \(RL\), the model could solve the tasks, but often did so with unnatural movements that did not resemble the motion of real rats\[[4](https://arxiv.org/html/2610.27279#bib.bib4)\]\. While researchers have since used imitation learning to improve the realism of virtual animal motion, these models are either limited to pure imitation\[[5](https://arxiv.org/html/2610.27279#bib.bib5)\], or struggle to transfer to downstream RL tasks\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\]\.

Within robotics, physics\-based controllers trained with RL to produce realistic humanoid motion can generalize to new tasks\[[7](https://arxiv.org/html/2610.27279#bib.bib7),[8](https://arxiv.org/html/2610.27279#bib.bib8),[9](https://arxiv.org/html/2610.27279#bib.bib9),[10](https://arxiv.org/html/2610.27279#bib.bib10),[11](https://arxiv.org/html/2610.27279#bib.bib11),[12](https://arxiv.org/html/2610.27279#bib.bib12)\]\. However, most approaches have no mechanism for random generation of realistic motion\. One exception is the recent Physics\-based Universal motion Latent SpacE \(PULSE\) framework that uses residual control of a variational motion prior\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\]\. PULSE can sample from its motion prior to produce random, yet realistic movements\. However, this sampling exhibits limited diversity: depending on initialization, the system typically settles into a single behavioral mode and stays there\. This is especially problematic when PULSE is used as a low\-level controller for sparse\-reward tasks, which demand substantial structured exploration\[[14](https://arxiv.org/html/2610.27279#bib.bib14),[15](https://arxiv.org/html/2610.27279#bib.bib15)\]\. Because most experimental tasks in neuroscience offer only sparse extrinsic reward\[[16](https://arxiv.org/html/2610.27279#bib.bib16),[17](https://arxiv.org/html/2610.27279#bib.bib17),[18](https://arxiv.org/html/2610.27279#bib.bib18)\], training embodied animal models to solve them while maintaining naturalistic motion remains an open problem\.

Here, we presentSimulatedControl viaAugmentedMotorPriors forEmbodiedReinforcement learning \(SCAMPER\), a system for training an embodied virtual rat model, enabling natural movements during both goal\-directed behavior and random exploration\. SCAMPER builds on the PULSE framework\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\], but adds an additional behavior prior that greatly increases the diversity of randomly generated movements while preserving the realism of the motion\. We provide a formal analysis demonstrating why the behavior prior increases the diversity of generated movements\. We then show empirically that SCAMPER produces a diverse array of realistic behavior during random rollouts, covering the behavioral repertoire present in the training data and beyond\. We also demonstrate that SCAMPER transfers effectively to a range of downstream tasks, from dense\-reward locomotion and rearing to sparse\-reward foraging tasks, all while maintaining naturalistic motion throughout training\. Altogether, SCAMPER provides the neuro\-AI community with a powerful new tool for embodied modeling of animal behavior\.

![Refer to caption](https://arxiv.org/html/2610.27279v1/fig1.png)Figure 1:Training stages of SCAMPER\.\(1\) The expert imitator is trained with mocap data\. \(2\) The latent controller is trained via online distillation\. \(3\) The behavior prior is distilled from the frozen encoder\. Finally, \(4\) the model can be trained with RL to perform different tasks\.
## 2Related work

Motor priors and latent skill spaces for physics\-based control\.Training physics\-based controllers from scratch for different tasks often yields unnatural motion\[[19](https://arxiv.org/html/2610.27279#bib.bib19)\], motivating imitation\-based controllers that track reference motion data\[[7](https://arxiv.org/html/2610.27279#bib.bib7)\]\.[Merel et al\. \[8\]](https://arxiv.org/html/2610.27279#bib.bib8)extended this approach by distilling a tracking expert into a latent skill space through an information bottleneck, and subsequent work explored adversarial alternatives\[[9](https://arxiv.org/html/2610.27279#bib.bib9),[10](https://arxiv.org/html/2610.27279#bib.bib10),[12](https://arxiv.org/html/2610.27279#bib.bib12)\]and variational formulations\[[20](https://arxiv.org/html/2610.27279#bib.bib20),[21](https://arxiv.org/html/2610.27279#bib.bib21),[22](https://arxiv.org/html/2610.27279#bib.bib22)\]\. PULSE\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\]distills a universal motion imitator\[[23](https://arxiv.org/html/2610.27279#bib.bib23)\]into a variational model regularized with a learnable motion prior, achieving broad coverage of human motor skills\. These frameworks target humanoid characters trained on human motion capture \(mocap\) data; their applicability to non\-human morphologies driven by species\-specific behavioral data has not yet been demonstrated\. Moreover, the learned motion prior in PULSE supports local motion continuation, but random rollouts tend to remain within the behavioral mode in which they are initialized, producing locally plausible but behaviorally narrow motion\. Kinematic motion models, including autoregressive CVAE\-based priors\[[24](https://arxiv.org/html/2610.27279#bib.bib24)\]and diffusion\-based generators\[[25](https://arxiv.org/html/2610.27279#bib.bib25)\], can produce diverse long\-horizon motion but do not yield a physics\-based controller\. Furthermore, autoregressive policies have been applied to physics\-based control\[[26](https://arxiv.org/html/2610.27279#bib.bib26),[27](https://arxiv.org/html/2610.27279#bib.bib27)\], but they provide no mechanism for sampling diverse motion that respects a species\-specific behavioral repertoire\. Here, we extend the PULSE framework to a biomechanically realistic rodent trained on 3D kinematic recordings of freely behaving rats, and complement the motion prior with a behavior prior that induces realistic transitions across behavioral modes\.

Physics\-based control of virtual animals\.Biomechanically realistic virtual animals have emerged as a tool for neuro\-AI\.[Merel et al\. \[4\]](https://arxiv.org/html/2610.27279#bib.bib4)introduced a virtual rodent trained with RL, but without imitation pretraining the resulting motion was unnatural\.[Aldarondo et al\. \[5\]](https://arxiv.org/html/2610.27279#bib.bib5)extended this work by training a controller for the virtual rat to imitate freely moving rats and found that its internal representations matched those in the motor cortex and dorsolateral striatum of the real rats; the controller was, however, purely imitative and could not generate diverse behavior\. MIMIC\-MJX\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\]generalizes this tracking\-imitation approach across species via an open\-source JAX/MJX pipeline, but supports only limited transfer learning\. Parallel efforts inDrosophila\[[28](https://arxiv.org/html/2610.27279#bib.bib28),[29](https://arxiv.org/html/2610.27279#bib.bib29)\]andC\. elegans\[[30](https://arxiv.org/html/2610.27279#bib.bib30)\]further extend this paradigm beyond rodents\. In zebrafish,[Keller et al\. \[31\]](https://arxiv.org/html/2610.27279#bib.bib31)drive naturalistic behavioral transitions via model\-based intrinsic motivation against an ethological prior\. Across this line of work, no existing model provides the ability to both mimic real motion and support sampling appropriate for exploration\. Our work fills this gap, yielding a controller that can be both reused across tasks and sampled for diverse, naturalistic behavior\.

## 3Overview of SCAMPER

### 3\.1Preliminaries

We define the proprioception of the agent at a given timestep as

𝐱t=\(𝜽t,𝜽˙t,𝝉t,ht,ϕt,𝐩te​e,𝐬tk​i​n,𝐜t,𝐚t−1\),\\mathbf\{x\}\_\{t\}=\(\\boldsymbol\{\\theta\}\_\{t\},\\boldsymbol\{\\dot\{\\theta\}\}\_\{t\},\\boldsymbol\{\\tau\}\_\{t\},h\_\{t\},\\boldsymbol\{\\phi\}\_\{t\},\\mathbf\{p\}^\{ee\}\_\{t\},\\mathbf\{s\}\_\{t\}^\{kin\},\\mathbf\{c\}\_\{t\},\\mathbf\{a\}\_\{t\-1\}\),\(1\)containing the joint angles \(𝜽t\\boldsymbol\{\\theta\}\_\{t\}\), angular velocities \(𝜽˙t\\boldsymbol\{\\dot\{\\theta\}\}\_\{t\}\), joint torques \(𝝉t\\boldsymbol\{\\tau\}\_\{t\}\), body height \(hth\_\{t\}\), orientation relative to the world z\-axis \(ϕt\\boldsymbol\{\\phi\}\_\{t\}, akin to a vestibular signal\), egocentric end\-effector positions \(𝐩te​e\\mathbf\{p\}^\{ee\}\_\{t\}\), kinematic sensors \(accelerometer, velocimeter, gyro;𝐬tk​i​n\\mathbf\{s\}\_\{t\}^\{kin\}\), end\-effector touch sensors \(𝐜t\\mathbf\{c\}\_\{t\}\), and the previous action \(𝐚t−1\\mathbf\{a\}\_\{t\-1\}\)\. During imitation tasks, the goal of the agent at a given timestep is defined as

𝐠t=\(𝐫t\+1−𝐫t,𝐪t\+1⊖𝐪t,𝜽t\+1−𝜽t,𝐩t\+1s​e​g−𝐩ts​e​g\),\\mathbf\{g\}\_\{t\}=\(\\mathbf\{r\}\_\{t\+1\}\-\\mathbf\{r\}\_\{t\},\\mathbf\{q\}\_\{t\+1\}\\ominus\\mathbf\{q\}\_\{t\},\\boldsymbol\{\\theta\}\_\{t\+1\}\-\\boldsymbol\{\\theta\}\_\{t\},\\mathbf\{p\}^\{seg\}\_\{t\+1\}\-\\mathbf\{p\}^\{seg\}\_\{t\}\),\(2\)and specifies the displacement the agent must achieve, computed as differences between its current state and the target state at the next timestep in the root position \(𝐫t\\mathbf\{r\}\_\{t\}\), root orientation \(𝐪t\\mathbf\{q\}\_\{t\}; where⊖\\ominusis egocentric quaternion difference\), joint angles \(𝜽t\\boldsymbol\{\\theta\}\_\{t\}\), and egocentric Cartesian body segment positions \(𝐩ts​e​g\\mathbf\{p\}^\{seg\}\_\{t\}\)\. Target states are provided by a mocap dataset registered to the biomechanical rat model via inverse kinematics using STAC\-MJX\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\]\. During imitation the policy network is provided the next five goal states \(gt:t\+4g\_\{t:t\+4\}\)\. During transfer learning, the agent no longer receives a goal state,𝐠t\\mathbf\{g\}\_\{t\}, and instead observes task\-specific environmental observations𝐨t\\mathbf\{o\}\_\{t\}\(e\.g\., scent gradient\)\. In this work we use the terms ‘rat’, ‘rodent’, and ‘agent’ interchangeably\.

We frame each task \(aside from the distillation objectives\) as goal\-conditioned RL, formulated as a Markov Decision Processℳ=⟨𝒮,𝒜,𝒯,ℛ,γ⟩\\mathcal\{M\}=\\left<\\mathcal\{S\},\\mathcal\{A\},\\mathcal\{T\},\\mathcal\{R\},\\gamma\\right\>\. The statesst∈𝒮s\_\{t\}\\in\\mathcal\{S\}contain𝐱t\\mathbf\{x\}\_\{t\}and either𝐠t\\mathbf\{g\}\_\{t\}or𝐨t\\mathbf\{o\}\_\{t\}; all values are normalized by their running mean and variance\. Actions𝐚t∈𝒜\\mathbf\{a\}\_\{t\}\\in\\mathcal\{A\}are torques applied to the rodent’s actuated joints\.𝒯\\mathcal\{T\}andℛ\\mathcal\{R\}are the transition dynamics \(determined by the physics environment\) and the reward function, respectively\. We train policies with proximal policy optimization \(PPO\)\[[32](https://arxiv.org/html/2610.27279#bib.bib32)\]to maximize the discounted return𝔼⁡\[∑t=1Tγt−1​rt\]\\mathbb\{E\}\\left\[\\sum\_\{t=1\}^\{T\}\\gamma^\{t\-1\}r\_\{t\}\\right\]\.

### 3\.2Training pipeline

SCAMPER uses a four\-stage training pipeline \(Fig\.[1](https://arxiv.org/html/2610.27279#S1.F1)\)\. All networks are multi\-layer perceptrons \(MLPs\), but in principle other architectures could be used\.

\(1\) Expert imitator\.We train an expert imitatorπI​\(𝐚t\|𝐱t,𝐠t\)\\pi\_\{I\}\(\\mathbf\{a\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\)with PPO to reproduce the mocap trajectories \(SuppVid1; App\.[C](https://arxiv.org/html/2610.27279#A3)\)\.

\(2\) Online distillation\.Following the PULSE framework\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\], we distill the expert imitator into a latent skill controller\. Similar to PULSE, we find that training the latent skill controller using supervised distillation of a pre\-trained expert leads to better training stability than training the latent skill controller from scratch using RL, hence we adopt the same approach\. The controller consists of three jointly trained networks: an encoderqϕ​\(𝐳t\|𝐱t,𝐠t\)q\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\), a decoderpθ​\(𝐚t\|𝐱t,𝐳t\)p\_\{\\theta\}\(\\mathbf\{a\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{z\}\_\{t\}\), and a proprioception\-conditioned motion priorpψ​\(𝐳t\|𝐱t\)p\_\{\\psi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)over latent codes\. Training proceeds by rolling out the student controller on the imitation task in simulation, while querying the frozen expert,πI\\pi\_\{I\}, for target actions on visited states, following DAgger\[[33](https://arxiv.org/html/2610.27279#bib.bib33)\]\. The networks are jointly trained by minimizing:

ℒdistill=ℒDAgger\+λ​ℒKL\+α​ℒAR,\\mathcal\{L\}\_\{\\mathrm\{distill\}\}=\\mathcal\{L\}\_\{\\mathrm\{DAgger\}\}\+\\lambda\\mathcal\{L\}\_\{\\mathrm\{KL\}\}\+\\alpha\\mathcal\{L\}\_\{\\mathrm\{AR\}\},\(3\)whereℒDAgger=‖𝐚t−𝐚^t‖22\\mathcal\{L\}\_\{\\mathrm\{DAgger\}\}=\|\|\\mathbf\{a\}\_\{t\}\-\\mathbf\{\\hat\{a\}\}\_\{t\}\|\|\_\{2\}^\{2\}matches the expert’s target action𝐚t\\mathbf\{a\}\_\{t\}to the decoder’s prediction𝐚^t\\mathbf\{\\hat\{a\}\}\_\{t\},ℒKL=DKL\(qϕ\|\|pψ\)\\mathcal\{L\}\_\{\\mathrm\{KL\}\}=D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\|\|p\_\{\\psi\}\)regularizes the encoder toward the motion prior, andℒAR=‖𝝁te−𝝁t−1e‖22\\mathcal\{L\}\_\{\\mathrm\{AR\}\}=\|\|\\boldsymbol\{\\mu\}\_\{t\}^\{e\}\-\\boldsymbol\{\\mu\}\_\{t\-1\}^\{e\}\|\|\_\{2\}^\{2\}encourages temporal smoothness in the latent space\. The decoder and motion prior are retained for downstream use; the encoder is kept only for the next training stage\.λ\\lambdaandα\\alphaare hyperparameters \(Table[3](https://arxiv.org/html/2610.27279#A5.T3)\)\.

\(3\) Behavior prior distillation\.After online distillation, we train a second proprioception\-conditioned priorpφ​\(𝐳t\|𝐱t\)p\_\{\\varphi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)with all other networks frozen\. Rolling out the latent skill controller on the imitation task in simulation, we optimizepφp\_\{\\varphi\}by minimizing

ℒbehavior=DKL\(qϕ\(𝐳t\|𝐱t,𝐠t\)∥pφ\(𝐳t\|𝐱t\)\)\.\\mathcal\{L\}\_\{\\mathrm\{behavior\}\}=D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\)\\,\\\|\\,p\_\{\\varphi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)\)\.\(4\)By Proposition[2](https://arxiv.org/html/2610.27279#Thmproposition2), this ensurespφp\_\{\\varphi\}captures the full goal\-conditional variation in the encoder’s outputs\.

\(4\) Transfer learning\.For downstream tasks, a task policyπT\(𝜹t,βt\|𝐱t,𝐨t\)\\pi\_\{T\}\(\\boldsymbol\{\\delta\}\_\{t\},\\beta\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{o\}\_\{t\}\)is trained with PPO to produce a residual correction𝜹t\\boldsymbol\{\\delta\}\_\{t\}to the motion prior’s mean𝝁tm\\boldsymbol\{\\mu\}\_\{t\}^\{m\}\. Importantly, we adjudicate between the two priors \(motion and behavior\) using a continuous gating variableβt∈\[−1,1\]\\beta\_\{t\}\\in\[\-1,1\]that is binarized tobt∈\{0,1\}b\_\{t\}\\in\\\{0,1\\\}via hysteresis thresholding\. Thus, at each timestep,btb\_\{t\}selects between two latent codes:

𝐳t=\{𝝁tm\+𝜹tif​bt=1,𝝁tb\+ϵ​𝝈,ϵ∼𝓝⁡\(𝟎,𝟏\)if​bt=0,\\mathbf\{z\}\_\{t\}=\\begin\{cases\}\\boldsymbol\{\\mu\}\_\{t\}^\{m\}\+\\boldsymbol\{\\delta\}\_\{t\}&\\text\{if \}b\_\{t\}=1,\\\\ \\boldsymbol\{\\mu\}\_\{t\}^\{b\}\+\\boldsymbol\{\\epsilon\}\\boldsymbol\{\\sigma\},\\quad\\boldsymbol\{\\epsilon\}\\sim\\boldsymbol\{\\mathcal\{N\}\(0,1\)\}&\\text\{if \}b\_\{t\}=0,\\end\{cases\}\(5\)where𝝁tm\\boldsymbol\{\\mu\}\_\{t\}^\{m\}and𝝁tb\\boldsymbol\{\\mu\}\_\{t\}^\{b\}are the means ofpψ​\(𝐳t\|𝐱t\)p\_\{\\psi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)andpφ​\(𝐳t\|𝐱t\)p\_\{\\varphi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\), respectively, and𝝈\\boldsymbol\{\\sigma\}is a fixed noise scale\. The resulting latent code is passed through the frozen decoderpθp\_\{\\theta\}to produce the action\. Intuitively,bt=1b\_\{t\}=1engages goal\-directed residual control of the motion prior \(exploitation\), whilebt=0b\_\{t\}=0samples naturalistic motion from the behavior prior \(exploration\)\. BecauseπT\\pi\_\{T\}outputsβt\\beta\_\{t\}, the system learns when to explore and when to exploit\.

### 3\.3Formal analysis of the priors

Why does SCAMPER use both a motion prior,pψp\_\{\\psi\}, for goal\-directed motion and a behavior prior,pφp\_\{\\varphi\}, for exploration? Although bothpψp\_\{\\psi\}andpφp\_\{\\varphi\}are proprioception\-conditioned distributions over the same latent space, their training objectives differ\. We formalize this distinction in two propositions\.

###### Proposition 1\(Information bottleneck under joint training\)\.

When the encoderqϕ​\(𝐳\|𝐠,𝐱\)q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)and the state\-dependent priorpψ​\(𝐳\|𝐱\)p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)are trained jointly, minimizing their expected KL divergence is equivalent to the following optimization:

minϕ,ψ𝔼p⁡\(𝐱,𝐠\)\[DKL\(qϕ∥pψ\)\]=minϕ,ψ\[Iϕ\(Z;G∣X\)\+𝔼p⁡\(𝐱\)\[DKL\(mϕ∥pψ\)\]\],\\min\_\{\\phi,\\psi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\\parallel p\_\{\\psi\}\)\\right\]=\\min\_\{\\phi,\\psi\}\\left\[I\_\{\\phi\}\(Z;G\\mid X\)\+\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\\parallel p\_\{\\psi\}\)\\right\]\\right\],\(6\)wheremϕ​\(𝐳\|𝐱\)=𝔼p⁡\(𝐠\|𝐱\)​\[qϕ​\(𝐳\|𝐠,𝐱\)\]m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)=\\mathbb\{E\}\_\{p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\}\[q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\]is the marginal posterior induced by the encoder, which evolved during joint training\.

Joint training of the encoder and the motion prior thus not only forces the motion prior to approximate the marginal posterior, but also minimizes the encoder’s conditional mutual information between the latent states and the goals\. However, from equation \([3](https://arxiv.org/html/2610.27279#S3.E3)\) both of these terms which the KL loss decomposes into are weighted by the same hyperparameter \(λ\\lambda\)\. As a result, in practice, it is difficult to obtain a favorable balance between regularization via the bottleneck and the distortion of movement imitation\. We find that settingλ\\lambdain order to obtain the right balance limits how closely the motion prior can approximate the marginal posterior, yielding a motion prior concentrated on local motion continuation\. We also find that stochastic sampling of the motion prior with high noise can occasionally push the system to switch behavioral modes, but as we show below \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)\), the motion prior alone does not reproduce the rich pattern of behavioral transitions observed in the training data\. Despite all this, a motion prior focused on local motion continuation yields a network favorable for residual control\. The marginal posterior, by contrast, contains the more fine\-grained structure of the behavioral transitions\. Therefore, in order to have the practical ability to generate diverse behavior during exploration we need another prior thatonly approximates the marginal posterior, allowing us to decouple the training dynamics from minimizing the conditional mutual information\. We can achieve this by training a new prior, the behavior prior, with the frozen encoder\.

###### Proposition 2\(Marginal posterior approximation under frozen encoder\)\.

When the encoderqϕ​\(𝐳\|𝐠,𝐱\)q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)is frozen, minimizing the expected KL divergence between the encoder and a state\-dependent priorpφ​\(𝐳\|𝐱\)p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)is equivalent to

minφ𝔼p⁡\(𝐱,𝐠\)\[DKL\(qϕ∥pφ\)\]=Iϕ\(Z;G∣X\)\+minφ𝔼p⁡\(𝐱\)\[DKL\(mϕ∥pφ\)\],\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\\parallel p\_\{\\varphi\}\)\\right\]=I\_\{\\phi\}\(Z;G\\mid X\)\+\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\\parallel p\_\{\\varphi\}\)\\right\],\(7\)wheremϕ​\(𝐳\|𝐱\)=𝔼p⁡\(𝐠\|𝐱\)​\[qϕ​\(𝐳\|𝐠,𝐱\)\]m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)=\\mathbb\{E\}\_\{p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\}\[q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\]is the marginal posterior induced by the frozen encoder, held fixed during optimization\. The conditional mutual information,Iϕ​\(Z;G∣X\)I\_\{\\phi\}\(Z;G\\mid X\), does not depend on the optimization variableφ\\varphi; it is therefore an additive constant that sits outside the minimization\. Hence, the optimal behavior prior is the closest possible state\-conditional approximation to the marginal posteriormϕm\_\{\\phi\}\.

Therefore, by directly distilling the behavior prior from the frozen pre\-trained encoder, we are able to lower the KL divergence between it and the goal\-rich encoder to a greater extent than we could for the motion prior which was limited by the rate\-distortion tradeoff\. Specifically, we achieve a KL divergence of 5\.88 nats between the behavior prior and the encoder, vs 14\.35 nats for the motion prior and the encoder\. This allows the behavior prior to better capture the behavioral transitions in the training data\. However, since the behavior prior is more inclined to sample from transitions in behavior in its control signal, it is less stable when residually controlled than the motion prior \(Fig\.[13](https://arxiv.org/html/2610.27279#A8.F13), Table[11](https://arxiv.org/html/2610.27279#A8.T11)\)\. This motivates the use of the dual priors for a sparse\-reward exploration task in Section[4\.5](https://arxiv.org/html/2610.27279#S4.SS5)\. We provide proofs for these two propositions in App\.[A](https://arxiv.org/html/2610.27279#A1)and empirical justification in App\.[H\.1](https://arxiv.org/html/2610.27279#A8.SS1)\. By training the behavior prior in this way, we achieve a good approximation for the marginal posterior,mϕ​\(𝐳\|𝐱\)m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\), which can then be used for effective exploration with natural movements, as we show in the subsequent experiments\.

## 4Experiments

### 4\.1The behavior prior expands behavioral diversity beyond the motion prior

![Refer to caption](https://arxiv.org/html/2610.27279v1/fig2.png)Figure 2:SCAMPER is behaviorally diverse\.\(a\)Renderings of behavior\-prior rollouts at increasing sampling noise\.\(b\)Vendi score vs\. latent sampling noiseσ=exp⁡\(logvar/2\)\\sigma=\\exp\(\\text\{logvar\}/2\), for the behavior prior \(blue\), motion prior \(purple\), and the training data \(dashed gray\); data shown is bootstrapped means with 95% CIs\.\(c\)Per\-rollout net displacement acrossσ\\sigma; violins with interquartile lines, gray violin marks training data\.\(d\)Spatial occupancy atσ=0\.22\\sigma=0\.22shown as 2D histograms of body location \(log\-scaled frame counts\) for the training data, motion prior, and behavior prior \(see Fig\.[9](https://arxiv.org/html/2610.27279#A8.F9)for all noise levels\)\.We first verify Propositions[1](https://arxiv.org/html/2610.27279#Thmproposition1)and[2](https://arxiv.org/html/2610.27279#Thmproposition2): that sampling the behavior prior produces greater behavioral diversity than sampling the motion prior \(SuppVid\[2\-4\]; App\.[C](https://arxiv.org/html/2610.27279#A3)\)\. We initialized the rodent in a realistic pose, then iteratively sampled a latent from one prior, decoded it to an action, and stepped the simulation for 5 s of simulated time per rollout \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)a\)\. We collected 842 rollouts per noise level, comparing pure behavior\-prior sampling \(bt=0b\_\{t\}=0\) to pure motion\-prior sampling \(bt=1b\_\{t\}=1\); see App\.[F\.1](https://arxiv.org/html/2610.27279#A6.SS1)\.

We measured the Vendi score\[[34](https://arxiv.org/html/2610.27279#bib.bib34)\], which captures the effective number of distinct behavioral modes\. The Vendi score has been used to quantify trajectory and skill diversity in RL\[[35](https://arxiv.org/html/2610.27279#bib.bib35),[36](https://arxiv.org/html/2610.27279#bib.bib36)\]\. Across all noise levels, the Vendi score was significantly higher for the behavior prior than the motion prior \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)b\)\. Motion\-prior sampling \(similar to PULSE\) produced lower Vendi scores than the training data for allσ<1\\sigma<1\. At very high noise, motion\-prior diversity matched the training data but at the cost of motion realism \(Section[4\.2](https://arxiv.org/html/2610.27279#S4.SS2)\)\. In contrast, the behavior prior matched the training data diversity at low noise and exceeded it at high noise\. This provides a tunable knob for controlling the behavioral diversity of the agent\.

Next, to assess whether this diversity aids exploration, we measured the net distance traveled across sampling noise levels\. Mirroring the Vendi score results, motion\-prior sampling produced shorter trajectories than the training data at almost all noise levels \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)c\)\. Behavior\-prior sampling roughly matched the training data distance at low noise and exceeded it at high noise\. We also examined the spatial occupancy in the virtual arena\. In the training data, the rat was in a circular arena, and thus, the occupancy was restricted to a circular area \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)d,left\)\. Motion\-prior rollouts at low noise remained largely within this same area despite no physical barrier \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)d,middle\), demonstrating that the motion prior samples from a restricted distribution of movements seen in the training data\. The behavior prior, in contrast, explored beyond this region \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)d,right\)\.

### 4\.2The behavior prior preserves motion realism under stochastic sampling

Figure 3:SCAMPER preserves motion realism during random behavior\.\(a\)Kernel Inception Distance \(KID\) between random rollouts and the training data behavior distributions across sampling noiseσ=exp⁡\(logvar/2\)\\sigma=\\exp\(\\text\{logvar\}/2\), for the behavior prior \(blue\) and motion prior \(purple\)\. Data shown is means±\\pm1 s\.d\.\(b\)Empirical cumulative distribution function \(ECDF\) of root height atσ=0\.22\\sigma=0\.22, compared against the training data \(dashed gray\)\.To verify that the movements induced by the behavior prior during random exploration were natural we used the Kernel Inception Distance \(KID\), which measures the dissimilarity between real and generated probability distributions\[[37](https://arxiv.org/html/2610.27279#bib.bib37)\]and has been used to measure the realism of generative human motion\[[38](https://arxiv.org/html/2610.27279#bib.bib38)\]\. We trained a variational autoencoder \(VAE\) on the joint angle trajectories from the training data as the inception model, then computed the KID between the VAE features of the training and generated trajectories\. Across all noise levels, the KID was lower for the behavior prior than the motion prior \(Fig\.[3](https://arxiv.org/html/2610.27279#S4.F3)a\)\. This confirms that the behavior prior produces more natural movements during random generation than the motion prior\.

For a more concrete measure of motion realism, we examined the rearing behavior of the virtual rat by measuring the proportion of time spent at various body heights\. The behavior prior more closely matched the training data’s rearing distribution, while the motion prior diverged from it \(Fig\.[3](https://arxiv.org/html/2610.27279#S4.F3)b\)\. This held across different levels of noise \(Fig\.[10](https://arxiv.org/html/2610.27279#A8.F10)\), and was quantitatively confirmed using the Wasserstein distance of the rearing curves between the generated data and the training data \(Table[9](https://arxiv.org/html/2610.27279#A8.T9)\)\. Together with Section[4\.1](https://arxiv.org/html/2610.27279#S4.SS1), these results show that the behavior prior can generate diverse and realistic rat motion during random sampling\.

### 4\.3The motion prior helps produce realistic gaits at different speeds during transfer

![Refer to caption](https://arxiv.org/html/2610.27279v1/fig4.png)Figure 4:SCAMPER produces realistic gaits at varying speeds\.\(a\)Actual vs\. target speed for SCAMPER \(blue\), MIMIC\-MJX \(red\), and the from\-scratch model \(gray\); lines show means, shaded bands show±1\\pm 1s\.d\. across seeds, dashed line indicates perfect tracking\. The training\-data speed distribution is shown in purple on the right\.\(b\)Renderings of SCAMPER walking, trotting, and galloping at increasing target speeds\.\(c\)Stance\-phase plots for each limb \(RF, LF, RH, LH\) across walk, trot, and gallop gaits, comparing empirical rat data \(purple; data from\[[39](https://arxiv.org/html/2610.27279#bib.bib39)\]\) to SCAMPER \(blue\); bars show stance duration, error bars show variability across strides\.Having confirmed that the behavior prior enables diverse and realistic random generation, we next tested whether residual control of the motion prior enables effective transfer to RL tasks\. We began with a dense\-reward locomotion task: at every timestep, the agent received a reward that decreased linearly with the difference between the target and actual speed\. We trained separate policy networks for each target speed\. SCAMPER closely matched the target speed across a range of realistic rat locomotion speeds \(Fig\.[4](https://arxiv.org/html/2610.27279#S4.F4)a,blue line\) whereas MIMIC\-MJX failed to generalize beyond speeds seen in the training data \(Fig\.[4](https://arxiv.org/html/2610.27279#S4.F4)a,red line\)\. A model trained from scratch \(as in[Merel et al\. \[4\]](https://arxiv.org/html/2610.27279#bib.bib4)\) also matched target speeds but with deeply unnatural motion and higher variance in its performance \(Fig\.[4](https://arxiv.org/html/2610.27279#S4.F4)a,gray line\)\.

To better understand the resulting movement we examined the gaits at different speeds\. Qualitatively, SCAMPER produced distinct gaits resembling walking, trotting, and galloping in real rats \(Fig\.[4](https://arxiv.org/html/2610.27279#S4.F4)b\)\. Notably, the training data—collected from freely behaving rats in a circular arena—contained no sustained or high\-speed locomotion, and thus no trot or gallop\. The emergence of these gaits in SCAMPER suggests it generalizes out\-of\-distribution while maintaining natural locomotion\.

To verify this, we performed a quantitative gait analysis, as is standard in behavioral experiments\[[40](https://arxiv.org/html/2610.27279#bib.bib40),[41](https://arxiv.org/html/2610.27279#bib.bib41)\], comparing SCAMPER, MIMIC\-MJX, and the from\-scratch model against real rat data\[[39](https://arxiv.org/html/2610.27279#bib.bib39)\]\. For each gait \(walk, trot, gallop\), SCAMPER matched real rat profiles both qualitatively \(Fig\.[4](https://arxiv.org/html/2610.27279#S4.F4)c\) and quantitatively \(Table[1](https://arxiv.org/html/2610.27279#S4.T1)\)\. Gait profiles of MIMIC\-MJX and the from\-scratch model significantly diverged from the real rats \(Fig\.[13](https://arxiv.org/html/2610.27279#A8.F13)\)\. Notably, MIMIC\-MJX could not match target speeds above 0\.5 m/s, so the higher\-speed gaits \(trot, gallop\) never emerged\. Together, these results show that residual control of the motion prior enables effective task transfer with natural movement and out\-of\-distribution gait generalization \(see SuppVid\[5\-14\] for videos demonstrating these results; App\.[C](https://arxiv.org/html/2610.27279#A3)\)\.

Table 1:Per\-metric distance to empirical rat data, computed as the absolute difference in means divided by empirical standard deviation,\|μe​m​p−μv​i​r​t​u​a​l\|/σemp\|\\mu\_\{emp\}\-\\mu\_\{virtual\}\|/\\sigma\_\{\\text\{emp\}\}\(↓\\downarrow\)\. Values shown are the average across metrics in each category: Coordination \(temporal symmetry, inter\-limb phase couplings\), Pattern \(phase couplings, duty factors\), and All \(18 gait metrics\)\. \(See Table[11](https://arxiv.org/html/2610.27279#A8.T11)for all metrics\)
### 4\.4The motion prior expands the stability and precision of rearing behaviors

![Refer to caption](https://arxiv.org/html/2610.27279v1/fig5.png)Figure 5:SCAMPER expands the precision and stability of learnable rearing behaviors\.\(a\)Example rollouts of a walk\-then\-rear policy, with the target rearing height indicated by the red box, for policies trained using SCAMPER \(top\), MIMIC\-MJX \(middle\), and trained from scratch \(bottom\)\.\(b\)Task success rate over simulation steps for each method; lines show the mean across seeds and bands show±1\\pm 1s\.d\. Only the SCAMPER policy solves the task consistently within the training budget\.To verify that these results extend beyond locomotion, we tested a dense\-reward rearing task in which the virtual rat must lift its head to a target location \(Fig\.[5](https://arxiv.org/html/2610.27279#S4.F5)a\)\. The reward decreases linearly with the between the skull and the target, and a one\-time bonus reward is obtained for holding at the target for a full second\. We set the target at the maximum rearing height observed in the training data to test each model’s ability to use the full range of motion of the virtual body\.

SCAMPER eventually achieved a high success rate \(Fig\.[5](https://arxiv.org/html/2610.27279#S4.F5)b,blue line\)\. In contrast, MIMIC\-MJX rarely held the target height long enough \(Fig\.[5](https://arxiv.org/html/2610.27279#S4.F5)b,red line\), and the from\-scratch model never reached the target, unable to use the full dynamic range of the rat body in a stable manner \(Fig\.[5](https://arxiv.org/html/2610.27279#S4.F5)b,gray line\)\. Thus, SCAMPER enables effective task transfer with realistic motion for both locomotion and rearing \(see SuppVid\[15\-17\] for videos demonstrating these results; App\.[C](https://arxiv.org/html/2610.27279#A3)\)\.

### 4\.5Adaptive prior gating balances exploration and exploitation in a sparse\-reward foraging task

Finally, we tested whether SCAMPER’s gating mechanism could switch between the behavior and motion priors to balance exploration and exploitation in a sparse\-reward task\. We chose a foraging task relevant to behavioral neuroscience: the virtual rodent must explore a circular arena and navigate to goal locations to collect treats \(Fig\.[6](https://arxiv.org/html/2610.27279#S4.F6)a\)\. In each trial, the rodent spawned at the center of the arena\. To force exploration, treats spawned only outside a minimum radius\. Each treat emitted a Gaussian “scent” field, with the rodent receiving the local odor gradient as a state variable\. Collecting a treat \(head overlapping its location\) yielded reward and removed both the treat and its scent\.

We compared the full SCAMPER model \(Fig\.[6](https://arxiv.org/html/2610.27279#S4.F6)b,blue line\) against a motion\-prior\-only variant \(Fig\.[6](https://arxiv.org/html/2610.27279#S4.F6)b,purple line\), MIMIC\-MJX \(Fig\.[6](https://arxiv.org/html/2610.27279#S4.F6)b,red line\), and the from\-scratch model \(Fig\.[6](https://arxiv.org/html/2610.27279#S4.F6)b,gray line\)\. All models found treats at small spawn radii, where little exploration was required\. However, only the full SCAMPER model with the ability to switch to random generation via the behavior prior \(Fig\.[12](https://arxiv.org/html/2610.27279#A8.F12)\) was capable of solving the task when the minimum spawn radius was large and extensive exploration was necessary\. This demonstrates that combining the behavior and motion priors enables SCAMPER to train on realistic sparse\-reward tasks that require effective exploration \(see SuppVid\[18\-22\] for videos demonstrating these results; App\.[C](https://arxiv.org/html/2610.27279#A3)\)\.

![Refer to caption](https://arxiv.org/html/2610.27279v1/fig6.png)Figure 6:SCAMPER enables efficient foraging under increasingly sparse reward\.\(a\)Arena forage task: treats \(blue circles\) are spawned uniformly within an annulus bounded by the minimum spawn radius \(red dashed\) and the arena boundary \(orange\), with a scent field \(colorbar\) guiding the rodent from its start position \(white\)\.\(b\)Number of treats collected per episode as a function of the minimum spawn radius, for policies trained using SCAMPER \(blue\), a motion\-prior\-only variant \(purple\), MIMIC\-MJX \(red\), and trained from scratch \(gray\)\. Lines show the mean across seeds and bands show±1\\pm 1s\.d\.

## 5Discussion

Summary\.In this paper, we introduced SCAMPER, a system for training a realistic rat body on realistic behavioral tasks\. SCAMPER enables natural movement generation during both goal\-directed behavior and random exploration, thanks to its dual\-prior architecture\. We demonstrated that the behavior prior expands the diversity of randomly sampled motion, while preserving motion realism\. We also showed that through residual control of the motion prior, SCAMPER can produce realistic gaits at different speeds and fully leverage its rearing capabilities\. Furthermore, we demonstrated that by combining both priors the virtual rodent can solve more difficult sparse\-reward tasks that demand exploration\.

Significance for neuro\-AI\.SCAMPER is designed primarily as a tool for neuro\-AI\. It builds on established techniques for imitation, latent skill learning, and residual control; its additions, a behavior prior distilled from the frozen encoder and a learned gate between the two priors, extend naturalistic motion from goal\-directed control to exploration\. Naturalistic exploration matters for neuroscience: most experimental tasks offer only sparse reward, so models must explore to learn them, and unnatural movement would confound comparisons with neural data\.[Aldarondo et al\. \[5\]](https://arxiv.org/html/2610.27279#bib.bib5)found that the internal representations of a purely imitative virtual rat matched neural activity in rat motor cortex and dorsolateral striatum\. SCAMPER makes it possible to extend such comparisons to goal\-directed tasks, with models and animals trained on the same task and compared throughout learning\.

Limitations & next steps\.There are several limitations to this work that should be recognized\. First and foremost, the dataset that we used for training the model was very limited\. It consisted of only 1 hour of a rodent freely behaving in an open arena\. As such, it contained only simple behaviors, namely walking, rearing, sitting still, and grooming\. A more extensive dataset with rats engaging in more diverse behaviors, exhibiting longer bouts of consistent locomotion at various speeds, and movement in more diverse landscapes, would be of great benefit\. Another key, but missing, component in the system we presented here is the presence of rich, multi\-modal sensory inputs\. The current version of SCAMPER uses only proprioceptive inputs, and a simple proxy for scent, effectively making the rat blind and deaf\. As the field moves towards more sophisticated embodied models and experiments, this limitation will become more pressing to address\. Future work can expand SCAMPER with a larger array of realistic sensory inputs\. Finally, one key difference between SCAMPER and real motion control in animals is the use of torque actuators rather than muscles\. Real animals must learn to control their bodies through activation of muscle fibers, and a good embodied model would likely take this facet of animal behavior into consideration\. In principle, SCAMPER is agnostic to the actuator type, so as new virtual bodies get developed, SCAMPER could be applied to muscle\-based models\.

## Acknowledgments and Disclosure of Funding

The authors would like to thank Nanda Krishna, João Sacramento and Guillaume Lajoie for discussions, support, and feedback\. AS was supported by a Fonds de recherche du Québec – Nature et technologies \(FRQNT\) Master’s Research Scholarship, a Natural Sciences and Engineering Research Council of Canada \(NSERC\) Canada Graduate Research Scholarship – Master’s \(CGRS M\), and a UNIQUE Excellence Scholarship\. CYZ was supported by the NSF CSGrad4US Fellowship\. This work was also supported by a L\.I\.F\.E\. Foundation grant to TDP, a Salk Innovation Award to TDP, an NIH BRAIN Initiative grant \(U01NS136507\) to BPÖ and TDP, a SFARI Investigator Award to BPÖ, and an ARNI award to BPÖ and BAR\. This research was enabled in part by compute resources provided by Mila \([https://mila\.quebec](https://mila.quebec/)\), Calcul Québec \([https://www\.calculquebec\.ca](https://www.calculquebec.ca/)\), and the Digital Research Alliance of Canada \([https://alliancecan\.ca](https://alliancecan.ca/)\)\.

## Competing interests

BAR is a Research Scientist at Google and AS is a Student Researcher at Google\. The authors declare no other competing interests\.

## References

- \[1\]Melanie Segado, Felipe Parodi, Jordan K\. Matelsky, Michael L\. Platt, Eva B\. Dyer, and Konrad P\. Kording\.Grounding Intelligence in Movement, July 2025\.URL[http://arxiv\.org/abs/2507\.02771](http://arxiv.org/abs/2507.02771)\.arXiv:2507\.02771 \[cs\]\.
- \[2\]Anthony Zador, Sean Escola, Blake Richards, Bence Ölveczky, Yoshua Bengio, Kwabena Boahen, Matthew Botvinick, Dmitri Chklovskii, Anne Churchland, Claudia Clopath, James DiCarlo, Surya Ganguli, Jeff Hawkins, Konrad Körding, Alexei Koulakov, Yann LeCun, Timothy Lillicrap, Adam Marblestone, Bruno Olshausen, Alexandre Pouget, Cristina Savin, Terrence Sejnowski, Eero Simoncelli, Sara Solla, David Sussillo, Andreas S\. Tolias, and Doris Tsao\.Catalyzing next\-generation Artificial Intelligence through NeuroAI\.*Nature Communications*, 14\(1\):1597, March 2023\.ISSN 2041\-1723\.doi:10\.1038/s41467\-023\-37180\-x\.URL[https://www\.nature\.com/articles/s41467\-023\-37180\-x](https://www.nature.com/articles/s41467-023-37180-x)\.
- \[3\]Sibo Wang\-Chen and Pavan Ramdya\.The embodied brain: Bridging the brain, body, and behavior with neuromechanical digital twins, January 2026\.URL[http://arxiv\.org/abs/2601\.08056](http://arxiv.org/abs/2601.08056)\.arXiv:2601\.08056 \[q\-bio\]\.
- \[4\]Josh Merel, Diego Aldarondo, Jesse Marshall, Yuval Tassa, Greg Wayne, and Bence Ölveczky\.Deep neuroethology of a virtual rodent, November 2019a\.URL[http://arxiv\.org/abs/1911\.09451](http://arxiv.org/abs/1911.09451)\.Issue: arXiv:1911\.09451 arXiv:1911\.09451 \[q\-bio\]\.
- \[5\]Diego Aldarondo, Josh Merel, Jesse D\. Marshall, Leonard Hasenclever, Ugne Klibaite, Amanda Gellis, Yuval Tassa, Greg Wayne, Matthew Botvinick, and Bence P\. Ölveczky\.A virtual rodent predicts the structure of neural activity across behaviours\.*Nature*, 632\(8025\):594–602, August 2024\.ISSN 0028\-0836, 1476\-4687\.doi:10\.1038/s41586\-024\-07633\-4\.URL[https://www\.nature\.com/articles/s41586\-024\-07633\-4](https://www.nature.com/articles/s41586-024-07633-4)\.Number: 8025\.
- \[6\]Charles Y\. Zhang, Yuanjia Yang, Aidan Sirbu, Elliott T\. T\. Abe, Emil Wärnberg, Eric J\. Leonardis, Diego E\. Aldarondo, Adam Lee, Aaditya Prasad, Jason Foat, Kaiwen Bian, Joshua Park, Rusham Bhatt, Hutton Saunders, Akira Nagamori, Ayesha R\. Thanawalla, Kee Wui Huang, Fabian Plum, Hendrik K\. Beck, Steven W\. Flavell, David Labonte, Blake A\. Richards, Bingni W\. Brunton, Eiman Azim, Bence P\. Ölveczky, and Talmo D\. Pereira\.MIMIC\-MJX: Neuromechanical Emulation of Animal Behavior, November 2025\.URL[http://arxiv\.org/abs/2511\.20532](http://arxiv.org/abs/2511.20532)\.arXiv:2511\.20532 \[q\-bio\]\.
- \[7\]Xue Bin Peng, Pieter Abbeel, Sergey Levine, and Michiel van de Panne\.DeepMimic: Example\-Guided Deep Reinforcement Learning of Physics\-Based Character Skills\.*ACM Transactions on Graphics*, 37\(4\):1–14, August 2018a\.ISSN 0730\-0301, 1557\-7368\.doi:10\.1145/3197517\.3201311\.URL[http://arxiv\.org/abs/1804\.02717](http://arxiv.org/abs/1804.02717)\.arXiv:1804\.02717 \[cs\]\.
- \[8\]Josh Merel, Leonard Hasenclever, Alexandre Galashov, Arun Ahuja, Vu Pham, Greg Wayne, Yee Whye Teh, and Nicolas Heess\.Neural probabilistic motor primitives for humanoid control, January 2019b\.URL[http://arxiv\.org/abs/1811\.11711](http://arxiv.org/abs/1811.11711)\.arXiv:1811\.11711 \[cs\]\.
- \[9\]Xue Bin Peng, Ze Ma, Pieter Abbeel, Sergey Levine, and Angjoo Kanazawa\.AMP: Adversarial Motion Priors for Stylized Physics\-Based Character Control\.*ACM Transactions on Graphics*, 40\(4\):1–20, August 2021\.ISSN 0730\-0301, 1557\-7368\.doi:10\.1145/3450626\.3459670\.URL[http://arxiv\.org/abs/2104\.02180](http://arxiv.org/abs/2104.02180)\.arXiv:2104\.02180 \[cs\]\.
- \[10\]Xue Bin Peng, Yunrong Guo, Lina Halper, Sergey Levine, and Sanja Fidler\.ASE: Large\-Scale Reusable Adversarial Skill Embeddings for Physically Simulated Characters\.*ACM Transactions on Graphics*, 41\(4\):1–17, July 2022\.ISSN 0730\-0301, 1557\-7368\.doi:10\.1145/3528223\.3530110\.URL[http://arxiv\.org/abs/2205\.01906](http://arxiv.org/abs/2205.01906)\.arXiv:2205\.01906 \[cs\]\.
- \[11\]Leonard Hasenclever, Fabio Pardo, Raia Hadsell, Nicolas Heess, and Josh Merel\.CoMic: Complementary Task Learning & Mimicry for Reusable Skills\.In*Proceedings of the 37th International Conference on Machine Learning*, pages 4105–4115\. PMLR, November 2020\.URL[https://proceedings\.mlr\.press/v119/hasenclever20a\.html](https://proceedings.mlr.press/v119/hasenclever20a.html)\.
- \[12\]Chen Tessler, Yoni Kasten, Yunrong Guo, Shie Mannor, Gal Chechik, and Xue Bin Peng\.CALM: Conditional Adversarial Latent Models for Directable Virtual Characters\.In*Special Interest Group on Computer Graphics and Interactive Techniques Conference Conference Proceedings*, pages 1–9, July 2023\.doi:10\.1145/3588432\.3591541\.URL[http://arxiv\.org/abs/2305\.02195](http://arxiv.org/abs/2305.02195)\.arXiv:2305\.02195 \[cs\]\.
- \[13\]Zhengyi Luo, Jinkun Cao, Josh Merel, Alexander Winkler, Jing Huang, Kris Kitani, and Weipeng Xu\.Universal Humanoid Motion Representations for Physics\-Based Control, April 2024\.URL[http://arxiv\.org/abs/2310\.04582](http://arxiv.org/abs/2310.04582)\.arXiv:2310\.04582 \[cs\]\.
- \[14\]Deepak Pathak, Pulkit Agrawal, Alexei A\. Efros, and Trevor Darrell\.Curiosity\-driven Exploration by Self\-supervised Prediction, May 2017\.URL[http://arxiv\.org/abs/1705\.05363](http://arxiv.org/abs/1705.05363)\.arXiv:1705\.05363 \[cs\]\.
- \[15\]Adrien Ecoffet, Joost Huizinga, Joel Lehman, Kenneth O\. Stanley, and Jeff Clune\.First return, then explore\.*Nature*, 590\(7847\):580–586, February 2021\.ISSN 0028\-0836, 1476\-4687\.doi:10\.1038/s41586\-020\-03157\-9\.URL[http://arxiv\.org/abs/2004\.12919](http://arxiv.org/abs/2004.12919)\.arXiv:2004\.12919 \[cs\]\.
- \[16\]Matteo Carandini and Anne K\. Churchland\.Probing perceptual decisions in rodents\.*Nature Neuroscience*, 16\(7\):824–831, July 2013\.ISSN 1546\-1726\.doi:10\.1038/nn\.3410\.URL[https://www\.nature\.com/articles/nn\.3410](https://www.nature.com/articles/nn.3410)\.
- \[17\]The International Brain Laboratory, Valeria Aguillon\-Rodriguez, Dora Angelaki, Hannah Bayer, Niccolo Bonacchi, Matteo Carandini, Fanny Cazettes, Gaelle Chapuis, Anne K Churchland, Yang Dan, Eric Dewitt, Mayo Faulkner, Hamish Forrest, Laura Haetzel, Michael Häusser, Sonja B Hofer, Fei Hu, Anup Khanal, Christopher Krasniak, Ines Laranjeira, Zachary F Mainen, Guido Meijer, Nathaniel J Miska, Thomas D Mrsic\-Flogel, Masayoshi Murakami, Jean\-Paul Noel, Alejandro Pan\-Vazquez, Cyrille Rossant, Joshua Sanders, Karolina Socha, Rebecca Terry, Anne E Urai, Hernando Vergara, Miles Wells, Christian J Wilson, Ilana B Witten, Lauren E Wool, and Anthony M Zador\.Standardized and reproducible measurement of decision\-making in mice\.*eLife*, 10:e63711, May 2021\.ISSN 2050\-084X\.doi:10\.7554/eLife\.63711\.URL[https://doi\.org/10\.7554/eLife\.63711](https://doi.org/10.7554/eLife.63711)\.
- \[18\]Simon Musall, Matthew T\. Kaufman, Ashley L\. Juavinett, Steven Gluf, and Anne K\. Churchland\.Single\-trial neural dynamics are dominated by richly varied movements\.*Nature Neuroscience*, 22\(10\):1677–1686, October 2019\.ISSN 1546\-1726\.doi:10\.1038/s41593\-019\-0502\-4\.URL[https://www\.nature\.com/articles/s41593\-019\-0502\-4](https://www.nature.com/articles/s41593-019-0502-4)\.
- \[19\]Nicolas Heess, Dhruva TB, Srinivasan Sriram, Jay Lemmon, Josh Merel, Greg Wayne, Yuval Tassa, Tom Erez, Ziyu Wang, S\. M\. Ali Eslami, Martin Riedmiller, and David Silver\.Emergence of Locomotion Behaviours in Rich Environments, July 2017\.URL[http://arxiv\.org/abs/1707\.02286](http://arxiv.org/abs/1707.02286)\.arXiv:1707\.02286 \[cs\]\.
- \[20\]Heyuan Yao, Zhenhua Song, Baoquan Chen, and Libin Liu\.ControlVAE: Model\-Based Learning of Generative Controllers for Physics\-Based Characters\.*ACM Transactions on Graphics*, 41\(6\):1–16, December 2022\.ISSN 0730\-0301, 1557\-7368\.doi:10\.1145/3550454\.3555434\.URL[http://arxiv\.org/abs/2210\.06063](http://arxiv.org/abs/2210.06063)\.arXiv:2210\.06063 \[cs\]\.
- \[21\]Jungdam Won, Deepak Gopinath, and Jessica Hodgins\.Physics\-based character controllers using conditional VAEs\.*ACM Trans\. Graph\.*, 41\(4\):96:1–96:12, July 2022\.ISSN 0730\-0301\.doi:10\.1145/3528223\.3530067\.URL[https://dl\.acm\.org/doi/10\.1145/3528223\.3530067](https://dl.acm.org/doi/10.1145/3528223.3530067)\.
- \[22\]Heyuan Yao, Zhenhua Song, Yuyang Zhou, Tenglong Ao, Baoquan Chen, and Libin Liu\.MoConVQ: Unified Physics\-Based Motion Control via Scalable Discrete Representations, December 2023\.URL[http://arxiv\.org/abs/2310\.10198](http://arxiv.org/abs/2310.10198)\.arXiv:2310\.10198 \[cs\]\.
- \[23\]Zhengyi Luo, Jinkun Cao, Alexander Winkler, Kris Kitani, and Weipeng Xu\.Perpetual Humanoid Control for Real\-time Simulated Avatars, September 2023\.URL[http://arxiv\.org/abs/2305\.06456](http://arxiv.org/abs/2305.06456)\.arXiv:2305\.06456 \[cs\]\.
- \[24\]Davis Rempe, Tolga Birdal, Aaron Hertzmann, Jimei Yang, Srinath Sridhar, and Leonidas J\. Guibas\.HuMoR: 3D Human Motion Model for Robust Pose Estimation, August 2021\.URL[http://arxiv\.org/abs/2105\.04668](http://arxiv.org/abs/2105.04668)\.arXiv:2105\.04668 \[cs\]\.
- \[25\]Guy Tevet, Sigal Raab, Brian Gordon, Yonatan Shafir, Daniel Cohen\-Or, and Amit H\. Bermano\.Human Motion Diffusion Model, October 2022\.URL[http://arxiv\.org/abs/2209\.14916](http://arxiv.org/abs/2209.14916)\.arXiv:2209\.14916 \[cs\]\.
- \[26\]Nolan Wagener, Andrey Kolobov, Felipe Vieira Frujeri, Ricky Loynd, Ching\-An Cheng, and Matthew Hausknecht\.MoCapAct: A Multi\-Task Dataset for Simulated Humanoid Control, January 2023\.URL[http://arxiv\.org/abs/2208\.07363](http://arxiv.org/abs/2208.07363)\.arXiv:2208\.07363 \[cs\]\.
- \[27\]Ilija Radosavovic, Bike Zhang, Baifeng Shi, Jathushan Rajasegaran, Sarthak Kamat, Trevor Darrell, Koushil Sreenath, and Jitendra Malik\.Humanoid Locomotion as Next Token Prediction, February 2024\.URL[http://arxiv\.org/abs/2402\.19469](http://arxiv.org/abs/2402.19469)\.arXiv:2402\.19469 \[cs\]\.
- \[28\]Roman Vaxenburg, Igor Siwanowicz, Josh Merel, Alice A\. Robie, Carmen Morrow, Guido Novati, Zinovia Stefanidi, Gert\-Jan Both, Gwyneth M\. Card, Michael B\. Reiser, Matthew M\. Botvinick, Kristin M\. Branson, Yuval Tassa, and Srinivas C\. Turaga\.Whole\-body physics simulation of fruit fly locomotion\.*Nature*, pages 1–3, April 2025\.ISSN 1476\-4687\.doi:10\.1038/s41586\-025\-09029\-4\.URL[https://www\.nature\.com/articles/s41586\-025\-09029\-4](https://www.nature.com/articles/s41586-025-09029-4)\.
- \[29\]Sibo Wang\-Chen, Victor Alfred Stimpfling, Thomas Ka Chung Lam, Pembe Gizem Özdil, Louise Genoud, Femke Hurtak, and Pavan Ramdya\.NeuroMechFly v2: simulating embodied sensorimotor control in adult Drosophila\.*Nature Methods*, 21\(12\):2353–2362, December 2024\.ISSN 1548\-7105\.doi:10\.1038/s41592\-024\-02497\-y\.URL[https://www\.nature\.com/articles/s41592\-024\-02497\-y](https://www.nature.com/articles/s41592-024-02497-y)\.
- \[30\]Mengdi Zhao, Ning Wang, Xinrui Jiang, Xiaoyang Ma, Haixin Ma, Gan He, Kai Du, Lei Ma, and Tiejun Huang\.An integrative data\-driven model simulating C\. elegans brain, body and environment interactions\.*Nature Computational Science*, 4\(12\):978–990, December 2024\.ISSN 2662\-8457\.doi:10\.1038/s43588\-024\-00738\-w\.URL[https://www\.nature\.com/articles/s43588\-024\-00738\-w](https://www.nature.com/articles/s43588-024-00738-w)\.
- \[31\]Reece Keller, Alyn Tornell, Felix Pei, Xaq Pitkow, Leo Kozachkov, and Aran Nayebi\.Autonomous Behavior and Whole\-Brain Dynamics Emerge in Embodied Zebrafish Agents with Model\-based Intrinsic Motivation, May 2025\.URL[http://arxiv\.org/abs/2506\.00138](http://arxiv.org/abs/2506.00138)\.arXiv:2506\.00138 \[q\-bio\]\.
- \[32\]John Schulman, Filip Wolski, Prafulla Dhariwal, Alec Radford, and Oleg Klimov\.Proximal Policy Optimization Algorithms, August 2017\.URL[http://arxiv\.org/abs/1707\.06347](http://arxiv.org/abs/1707.06347)\.arXiv:1707\.06347 \[cs\]\.
- \[33\]Stephane Ross, Geoffrey J\. Gordon, and J\. Andrew Bagnell\.A Reduction of Imitation Learning and Structured Prediction to No\-Regret Online Learning, March 2011\.URL[http://arxiv\.org/abs/1011\.0686](http://arxiv.org/abs/1011.0686)\.arXiv:1011\.0686 \[cs\]\.
- \[34\]Dan Friedman and Adji Bousso Dieng\.The Vendi Score: A Diversity Evaluation Metric for Machine Learning, July 2023\.URL[http://arxiv\.org/abs/2210\.02410](http://arxiv.org/abs/2210.02410)\.arXiv:2210\.02410 \[cs\]\.
- \[35\]Tom Stanic, Marco Jiralerspong, Xiaofeng Zhang, Danilo Vucetic, and Gauthier Gidel\.Beyond reward maximization: Evaluating the diversity of trajectories in reinforcement learning with temporal vendi score\.2026\.
- \[36\]Erik M\. Lintunen\.VendiRL: A Framework for Self\-Supervised Reinforcement Learning of Diversely Diverse Skills, October 2025\.URL[http://arxiv\.org/abs/2509\.02930](http://arxiv.org/abs/2509.02930)\.arXiv:2509\.02930 \[cs\]\.
- \[37\]Mikołaj Bińkowski, Danica J\. Sutherland, Michael Arbel, and Arthur Gretton\.Demystifying MMD GANs, January 2021\.URL[http://arxiv\.org/abs/1801\.01401](http://arxiv.org/abs/1801.01401)\.arXiv:1801\.01401 \[stat\]\.
- \[38\]Sigal Raab, Inbal Leibovitch, Peizhuo Li, Kfir Aberman, Olga Sorkine\-Hornung, and Daniel Cohen\-Or\.MoDi: Unconditional Motion Synthesis from Diverse Data, December 2022\.URL[http://arxiv\.org/abs/2206\.08010](http://arxiv.org/abs/2206.08010)\.arXiv:2206\.08010 \[cs\]\.
- \[39\]Simon Danner, Courtney Shepard, Casey Hainline, Natalia Shevtsova, Ilya Rybak, and David Magnuson\.*Spinal control of locomotion before and after spinal cord injury*\.June 2023\.doi:10\.1101/2023\.03\.22\.533794\.Journal Abbreviation: bioRxiv : the preprint server for biology Publication Title: bioRxiv : the preprint server for biology\.
- \[40\]Frank P\.T\. Hamers, Alex J\. Lankhorst, Teus Jan van Laar, Wouter B\. Veldhuis, and Willem Hendrik Gispen\.Automated Quantitative Gait Analysis During Overground Locomotion in the Rat: Its Application to Spinal Cord Contusion and Transection Injuries\.*Journal of Neurotrauma*, 18\(2\):187–201, February 2001\.ISSN 0897\-7151\.doi:10\.1089/08977150150502613\.URL[https://journals\.sagepub\.com/action/showAbstract](https://journals.sagepub.com/action/showAbstract)\.
- \[41\]César S\. Mendes, Imre Bartos, Zsuzsanna Márka, Turgay Akay, Szabolcs Márka, and Richard S\. Mann\.Quantification of gait parameters in freely walking rodents\.*BMC Biology*, 13\(1\):50, July 2015\.ISSN 1741\-7007\.doi:10\.1186/s12915\-015\-0154\-0\.URL[https://doi\.org/10\.1186/s12915\-015\-0154\-0](https://doi.org/10.1186/s12915-015-0154-0)\.
- \[42\]Timothy W Dunn, Jesse D Marshall, Kyle S Severson, Diego E Aldarondo, David G C Hildebrand, Selmaan N Chettih, William L Wang, Amanda J Gellis, David E Carlson, Dmitriy Aronov, Winrich A Freiwald, Fan Wang, and Bence P Ölveczky\.Geometric deep learning enables 3D kinematic profiling across species and environments\.*Nat\. Methods*, 18\(5\):564–573, May 2021\.
- \[43\]Emanuel Todorov, Tom Erez, and Yuval Tassa\.MuJoCo: A physics engine for model\-based control\.*2012 IEEE/RSJ International Conference on Intelligent Robots and Systems*, pages 5026–5033, October 2012\.
- \[44\]MuJoCo XLA Authors\.MuJoCo XLA \(MJX\)\.[https://mujoco\.readthedocs\.io/en/stable/mjx\.html](https://mujoco.readthedocs.io/en/stable/mjx.html)\.
- \[45\]James Bradbury, Roy Frostig, Peter Hawkins, Matthew James Johnson, Chris Leary, Dougal Maclaurin, George Necula, Adam Paszke, Jake VanderPlas, Skye Wanderman\-Milne, and Qiao Zhang\.JAX: composable transformations of Python\+NumPy programs, 2018\.
- \[46\]Kevin Zakka, Baruch Tabanpour, Qiayuan Liao, Mustafa Haiderbhai, Samuel Holt, Jing Yuan Luo, Arthur Allshire, Erik Frey, Koushil Sreenath, Lueder A Kahrs, Carmelo Sferrazza, Yuval Tassa, and Pieter Abbeel\.MuJoCo playground\.*arXiv \[cs\.RO\]*, February 2025\.
- \[47\]C Daniel Freeman, Erik Frey, Anton Raichuk, Sertan Girgin, Igor Mordatch, and Olivier Bachem\.Brax – a differentiable physics engine for large scale rigid body simulation\.*arXiv \[cs\.RO\]*, June 2021\.
- \[48\]Xue Bin Peng, Marcin Andrychowicz, Wojciech Zaremba, and Pieter Abbeel\.Sim\-to\-Real Transfer of Robotic Control with Dynamics Randomization\.In*2018 IEEE International Conference on Robotics and Automation \(ICRA\)*, pages 3803–3810, May 2018b\.doi:10\.1109/ICRA\.2018\.8460528\.URL[http://arxiv\.org/abs/1710\.06537](http://arxiv.org/abs/1710.06537)\.arXiv:1710\.06537 \[cs\]\.
- \[49\]Martin Heusel, Hubert Ramsauer, Thomas Unterthiner, Bernhard Nessler, and Sepp Hochreiter\.GANs Trained by a Two Time\-Scale Update Rule Converge to a Local Nash Equilibrium, January 2018\.URL[http://arxiv\.org/abs/1706\.08500](http://arxiv.org/abs/1706.08500)\.arXiv:1706\.08500 \[cs\]\.
- \[50\]Yi Zhou, Connelly Barnes, Jingwan Lu, Jimei Yang, and Hao Li\.On the Continuity of Rotation Representations in Neural Networks, June 2020\.URL[http://arxiv\.org/abs/1812\.07035](http://arxiv.org/abs/1812.07035)\.arXiv:1812\.07035 \[cs\]\.
- \[51\]Arthur Gretton, Karsten M\. Borgwardt, Malte J\. Rasch, Bernhard Schölkopf, and Alexander Smola\.A Kernel Two\-Sample Test\.*Journal of Machine Learning Research*, 13\(25\):723–773, 2012\.ISSN 1533\-7928\.URL[http://jmlr\.org/papers/v13/gretton12a\.html](http://jmlr.org/papers/v13/gretton12a.html)\.

## Appendix AProofs

We restate each proposition from Section[3\.3](https://arxiv.org/html/2610.27279#S3.SS3)and provide the full proof\.

###### Proposition 1\(Information bottleneck under joint training\)\.

When the encoderqϕ​\(𝐳\|𝐠,𝐱\)q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)and the state\-dependent priorpψ​\(𝐳\|𝐱\)p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)are trained jointly, minimizing their expected KL divergence decomposes as

minϕ,ψ𝔼p⁡\(𝐱,𝐠\)\[DKL\(qϕ∥pψ\)\]=minϕ,ψ\[Iϕ\(Z;G∣X\)\+𝔼p⁡\(𝐱\)\[DKL\(mϕ∥pψ\)\]\],\\min\_\{\\phi,\\psi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\\parallel p\_\{\\psi\}\)\\right\]=\\min\_\{\\phi,\\psi\}\\left\[I\_\{\\phi\}\(Z;G\\mid X\)\+\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\\parallel p\_\{\\psi\}\)\\right\]\\right\],\(8\)wheremϕ​\(𝐳\|𝐱\)=𝔼p⁡\(𝐠\|𝐱\)​\[qϕ​\(𝐳\|𝐠,𝐱\)\]m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)=\\mathbb\{E\}\_\{p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\}\[q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\]is the marginal posterior induced by the encoder, which evolved during joint training\.

###### Proof\.

Let the joint optimization objective be the expected KL divergence over the data distribution:

minϕ,ψℒ\(ϕ,ψ\)=minϕ,ψ𝔼p⁡\(𝐱,𝐠\)\[DKL\(qϕ\(𝐳\|𝐠,𝐱\)∥pψ\(𝐳\|𝐱\)\)\]\.\\min\_\{\\phi,\\psi\}\\mathcal\{L\}\(\\phi,\\psi\)=\\min\_\{\\phi,\\psi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\parallel p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\big\)\\right\]\.\(9\)Expanding the expected KL divergence into its integral form:

minϕ,ψ⁡ℒ⁡\(ϕ,ψ\)=min⁡∫ϕ,ψ⁡p⁡\(𝐱\)​∫p⁡\(𝐠\|𝐱\)​∫qϕ​\(𝐳\|𝐠,𝐱\)​log​qϕ​\(𝐳\|𝐠,𝐱\)pψ​\(𝐳\|𝐱\)​𝑑𝐳​𝑑𝐠​𝑑𝐱\.\\min\_\{\\phi,\\psi\}\\mathcal\{L\}\(\\phi,\\psi\)=\\min\_\{\\phi,\\psi\}\\int p\(\\mathbf\{x\}\)\\int p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\log\\frac\{q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\,d\\mathbf\{z\}\\,d\\mathbf\{g\}\\,d\\mathbf\{x\}\.\(10\)Define the marginal posterior \(which depends on the encoder parametersϕ\\phi\):

mϕ​\(𝐳\|𝐱\)≜∫qϕ​\(𝐳\|𝐠,𝐱\)​p​\(𝐠\|𝐱\)​𝑑𝐠\.m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\triangleq\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\,d\\mathbf\{g\}\.\(11\)Multiplying numerator and denominator inside the logarithm bymϕ​\(𝐳\|𝐱\)m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\):

log⁡qϕ​\(𝐳\|𝐠,𝐱\)pψ​\(𝐳\|𝐱\)=log⁡qϕ​\(𝐳\|𝐠,𝐱\)pψ​\(𝐳\|𝐱\)​mϕ​\(𝐳\|𝐱\)mϕ​\(𝐳\|𝐱\)=log⁡qϕ​\(𝐳\|𝐠,𝐱\)mϕ​\(𝐳\|𝐱\)\+log⁡mϕ​\(𝐳\|𝐱\)pψ​\(𝐳\|𝐱\)\.\\log\\frac\{q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}=\\log\\frac\{q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\frac\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}=\\log\\frac\{q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\}\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\+\\log\\frac\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\.\(12\)Substituting this decomposition back into the integral yields two terms:

minϕ,ψ⁡ℒ⁡\(ϕ,ψ\)\\displaystyle\\min\_\{\\phi,\\psi\}\\mathcal\{L\}\(\\phi,\\psi\)=minϕ,ψ\[∫p⁡\(𝐱\)​∫p⁡\(𝐠\|𝐱\)​∫qϕ​\(𝐳\|𝐠,𝐱\)​log⁡qϕ​\(𝐳\|𝐠,𝐱\)mϕ​\(𝐳\|𝐱\)​d𝐳​d𝐠​d𝐱⏟Term 1\\displaystyle=\\min\_\{\\phi,\\psi\}\\left\[\\underbrace\{\\int p\(\\mathbf\{x\}\)\\int p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\log\\frac\{q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\}\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\,d\\mathbf\{z\}\\,d\\mathbf\{g\}\\,d\\mathbf\{x\}\}\_\{\\text\{Term 1\}\}\\right\.\(13\)\+∫p⁡\(𝐱\)​∫p⁡\(𝐠\|𝐱\)​∫qϕ​\(𝐳\|𝐠,𝐱\)​log⁡mϕ​\(𝐳\|𝐱\)pψ​\(𝐳\|𝐱\)​d𝐳​d𝐠​d𝐱⏟Term 2\]\.\\displaystyle\\left\.\\quad\+\\underbrace\{\\int p\(\\mathbf\{x\}\)\\int p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\log\\frac\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\,d\\mathbf\{z\}\\,d\\mathbf\{g\}\\,d\\mathbf\{x\}\}\_\{\\text\{Term 2\}\}\\right\]\.
Term 1is, by definition, the conditional mutual information between the latent𝐳\\mathbf\{z\}and the goal𝐠\\mathbf\{g\}given the state𝐱\\mathbf\{x\}:

Term 1=Iϕ​\(Z;G∣X\)\.\\text\{Term 1\}=I\_\{\\phi\}\(Z;G\\mid X\)\.\(14\)
Term 2: Sincelog⁡mϕ​\(𝐳\|𝐱\)pψ​\(𝐳\|𝐱\)\\log\\frac\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}is independent of𝐠\\mathbf\{g\}, and the integrand is absolutely integrable111Absolute integrability follows from the assumption thatDKL\(qϕ\|\|pψ\)<∞D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\|\|p\_\{\\psi\}\)<\\inftyon the support ofp⁡\(𝐱,𝐠\)p\(\\mathbf\{x\},\\mathbf\{g\}\)\., we apply Fubini’s theorem to evaluate the integral over𝐠\\mathbf\{g\}first:

Term 2\\displaystyle\\text\{Term 2\}=∫p⁡\(𝐱\)​∫\(∫qϕ​\(𝐳\|𝐠,𝐱\)​p​\(𝐠\|𝐱\)​d𝐠\)⏟mϕ​\(𝐳\|𝐱\)​log⁡mϕ​\(𝐳\|𝐱\)pψ​\(𝐳\|𝐱\)​𝐝𝐳​𝐝𝐱\\displaystyle=\\int p\(\\mathbf\{x\}\)\\int\\underbrace\{\\left\(\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\,d\\mathbf\{g\}\\right\)\}\_\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\log\\frac\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\{p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\,d\\mathbf\{z\}\\,d\\mathbf\{x\}\(15\)=𝔼p⁡\(𝐱\)\[DKL\(mϕ\(𝐳\|𝐱\)∥pψ\(𝐳\|𝐱\)\)\]\.\\displaystyle=\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\parallel p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\big\)\\right\]\.
Combining the two terms gives the desired decomposition:

minϕ,ψℒ\(ϕ,ψ\)=minϕ,ψ\[Iϕ\(Z;G∣X\)\+𝔼p⁡\(𝐱\)\[DKL\(mϕ\(𝐳\|𝐱\)∥pψ\(𝐳\|𝐱\)\)\]\]\.\\min\_\{\\phi,\\psi\}\\mathcal\{L\}\(\\phi,\\psi\)=\\min\_\{\\phi,\\psi\}\\left\[I\_\{\\phi\}\(Z;G\\mid X\)\+\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\parallel p\_\{\\psi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\big\)\\right\]\\right\]\.\(16\)∎

###### Proposition 2\(Marginal posterior approximation under frozen encoder\)\.

When the encoderqϕ​\(𝐳\|𝐠,𝐱\)q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)is frozen, minimizing the expected KL divergence between the encoder and a state\-dependent priorpφ​\(𝐳\|𝐱\)p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)is equivalent to

minφ𝔼p⁡\(𝐱,𝐠\)\[DKL\(qϕ∥pφ\)\]=Iϕ\(Z;G∣X\)\+minφ𝔼p⁡\(𝐱\)\[DKL\(mϕ∥pφ\)\],\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\\parallel p\_\{\\varphi\}\)\\right\]=I\_\{\\phi\}\(Z;G\\mid X\)\+\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\\parallel p\_\{\\varphi\}\)\\right\],\(17\)wheremϕ​\(𝐳\|𝐱\)=𝔼p⁡\(𝐠\|𝐱\)​\[qϕ​\(𝐳\|𝐠,𝐱\)\]m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)=\\mathbb\{E\}\_\{p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\}\[q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\]is the marginal posterior induced by the frozen encoder, held fixed during optimization\. The conditional mutual information,Iϕ​\(Z;G∣X\)I\_\{\\phi\}\(Z;G\\mid X\), does not depend on the optimization variableφ\\varphi; it is therefore an additive constant that sits outside the minimization\. Hence, the optimal behavior prior is the closest state\-conditional approximation to the marginal posteriormϕm\_\{\\phi\}\.

###### Proof\.

Let the distillation objective be the expected KL divergence over the data distribution:

minφℒ\(φ\)=minφ𝔼p⁡\(𝐱,𝐠\)\[DKL\(qϕ\(𝐳\|𝐠,𝐱\)∥pφ\(𝐳\|𝐱\)\)\]\.\\min\_\{\\varphi\}\\mathcal\{L\}\(\\varphi\)=\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\\big\(q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\parallel p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\big\)\\right\]\.\(18\)Expanding the inner KL divergence as the difference between cross\-entropy and entropy:

DKL\(qϕ∥pφ\)=−∫qϕ\(𝐳\|𝐠,𝐱\)logpφ\(𝐳\|𝐱\)d𝐳⏟H⁡\(qϕ,pφ\)\+∫qϕ​\(𝐳\|𝐠,𝐱\)​log⁡qϕ​\(𝐳\|𝐠,𝐱\)​d𝐳⏟−H⁡\(qϕ\)\.D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\\parallel p\_\{\\varphi\}\)=\\underbrace\{\-\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\log p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\,d\\mathbf\{z\}\}\_\{H\(q\_\{\\phi\},p\_\{\\varphi\}\)\}\+\\underbrace\{\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\log q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\,d\\mathbf\{z\}\}\_\{\-H\(q\_\{\\phi\}\)\}\.\(19\)
Since the encoder parametersϕ\\phiare frozen, the entropyH⁡\(qϕ\)H\(q\_\{\\phi\}\)is constant with respect toφ\\varphiand can be pulled outside the minimization:

minφ⁡ℒ⁡\(φ\)=minφ⁡\[𝔼p⁡\(𝐱,𝐠\)​\[H⁡\(qϕ,pφ\)\]\]−𝔼p⁡\(𝐱,𝐠\)​\[H⁡\(qϕ\)\]\.\\min\_\{\\varphi\}\\mathcal\{L\}\(\\varphi\)=\\min\_\{\\varphi\}\\left\[\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\},p\_\{\\varphi\}\)\]\\right\]\-\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\}\)\]\.\(20\)First we focus on the first term inside the minimization and expand it out:

𝔼p⁡\(𝐱,𝐠\)\[H\(qϕ,pφ\)\]=−∫p\(𝐱\)∫p\(𝐠\|𝐱\)∫qϕ\(𝐳\|𝐠,𝐱\)logpφ\(𝐳\|𝐱\)d𝐳d𝐠d𝐱\.\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\},p\_\{\\varphi\}\)\]=\-\\int p\(\\mathbf\{x\}\)\\int p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)\\log p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\,d\\mathbf\{z\}\\,d\\mathbf\{g\}\\,d\\mathbf\{x\}\.\(21\)Assuming absolute integrability, we apply Fubini’s theorem to swap the order of integration between𝐠\\mathbf\{g\}and𝐳\\mathbf\{z\}, and factor outlog⁡pφ​\(𝐳\|𝐱\)\\log p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)since it is independent of𝐠\\mathbf\{g\}:

𝔼p⁡\(𝐱,𝐠\)​\[H⁡\(qϕ,pφ\)\]\\displaystyle\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\},p\_\{\\varphi\}\)\]=−∫p\(𝐱\)∫logpφ\(𝐳\|𝐱\)\(∫qϕ​\(𝐳\|𝐠,𝐱\)​p​\(𝐠\|𝐱\)​d𝐠\)⏟mϕ​\(𝐳\|𝐱\)d𝐳d𝐱\\displaystyle=\-\\int p\(\\mathbf\{x\}\)\\int\\log p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\underbrace\{\\left\(\\int q\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{g\},\\mathbf\{x\}\)p\(\\mathbf\{g\}\|\\mathbf\{x\}\)\\,d\\mathbf\{g\}\\right\)\}\_\{m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\}\\,d\\mathbf\{z\}\\,d\\mathbf\{x\}\(22\)=−∫p\(𝐱\)∫mϕ\(𝐳\|𝐱\)logpφ\(𝐳\|𝐱\)d𝐳d𝐱\\displaystyle=\-\\int p\(\\mathbf\{x\}\)\\int m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\log p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\,d\\mathbf\{z\}\\,d\\mathbf\{x\}=𝔼p⁡\(𝐱\)​\[H⁡\(mϕ,pφ\)\]\.\\displaystyle=\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\},p\_\{\\varphi\}\)\]\.This is the expectation of the cross\-entropy betweenmϕm\_\{\\phi\}andpφp\_\{\\varphi\}\. Substituting back into equation \([20](https://arxiv.org/html/2610.27279#A1.E20)\) yields:

minφ⁡ℒ⁡\(φ\)=minφ⁡\[𝔼p⁡\(𝐱\)​\[H⁡\(mϕ,pφ\)\]\]−𝔼p⁡\(𝐱,𝐠\)​\[H⁡\(qϕ\)\]\.\\min\_\{\\varphi\}\\mathcal\{L\}\(\\varphi\)=\\min\_\{\\varphi\}\\left\[\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\},p\_\{\\varphi\}\)\]\\right\]\-\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\}\)\]\.\(23\)
To relate it to the KL divergence, we add and subtract the expectation of the entropy of the marginal posterior:

minφ⁡ℒ⁡\(φ\)=minφ⁡\[𝔼p⁡\(𝐱\)​\[H⁡\(mϕ,pφ\)\]\]−𝔼p⁡\(𝐱\)​\[H⁡\(mϕ\)\]\+𝔼p⁡\(𝐱\)​\[H⁡\(mϕ\)\]−𝔼p⁡\(𝐱,𝐠\)​\[H⁡\(qϕ\)\]\.\\min\_\{\\varphi\}\\mathcal\{L\}\(\\varphi\)=\\min\_\{\\varphi\}\\left\[\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\},p\_\{\\varphi\}\)\]\\right\]\-\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\}\)\]\+\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\}\)\]\-\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\}\)\]\.\(24\)Pulling𝔼p⁡\(𝐱\)​\[H⁡\(mϕ\)\]\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\}\)\]into the minimization operation and using the identityH\(mϕ,pφ\)=DKL\(mϕ\|\|pφ\)\+H\(mϕ\)H\(m\_\{\\phi\},p\_\{\\varphi\}\)=D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\|\|p\_\{\\varphi\}\)\+H\(m\_\{\\phi\}\):

minφℒ\(φ\)=minφ𝔼p⁡\(𝐱\)\[DKL\(mϕ\(𝐳\|𝐱\)∥pφ\(𝐳\|𝐱\)\)\]\+\(𝔼p⁡\(𝐱\)\[H\(mϕ\)\]−𝔼p⁡\(𝐱,𝐠\)\[H\(qϕ\)\]\)\.\\min\_\{\\varphi\}\\mathcal\{L\}\(\\varphi\)=\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\parallel p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\)\\right\]\+\\Big\(\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\[H\(m\_\{\\phi\}\)\]\-\\mathbb\{E\}\_\{p\(\\mathbf\{x\},\\mathbf\{g\}\)\}\[H\(q\_\{\\phi\}\)\]\\Big\)\.\(25\)The bracketed difference of entropies is, by definition, the conditional mutual information between the latent and the goal given the state under the frozen encoder, therefore:

minφℒ\(φ\)=minφ𝔼p⁡\(𝐱\)\[DKL\(mϕ\(𝐳\|𝐱\)∥pφ\(𝐳\|𝐱\)\)\]\+Iϕ\(Z;G∣X\)\.\\min\_\{\\varphi\}\\mathcal\{L\}\(\\varphi\)=\\min\_\{\\varphi\}\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\)\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\\parallel p\_\{\\varphi\}\(\\mathbf\{z\}\|\\mathbf\{x\}\)\)\\right\]\+I\_\{\\phi\}\(Z;G\\mid X\)\.\(26\)∎

## Appendix BDataset

Motion capture\.We use a rat motion dataset collected by[Aldarondo et al\. \[5\]](https://arxiv.org/html/2610.27279#bib.bib5)\. The dataset is composed of 842 5 s clips of rats freely behaving in a circular enclosure\. The mocap data is characterized by 23 keypoints captured at 50 Hz using DANNCE v1\.3\[[42](https://arxiv.org/html/2610.27279#bib.bib42)\]from multicamera video\. The dataset contains various kinds of walking, rearing, grooming and prone behaviors\. The mocap keypoints are aligned to the virtual rodent body model through STAC\-MJX\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\]\.

Empirical rat locomotion data\.The empirical data used in the gait analysis \(phase differences, duty factors, and gait classifications\) were obtained from[Danner et al\. \[39\]](https://arxiv.org/html/2610.27279#bib.bib39), publicly available at[https://github\.com/dannerlab/rat\-sci\-locomotion](https://github.com/dannerlab/rat-sci-locomotion)under a Creative Commons Attribution–ShareAlike 4\.0 International license \(CC BY–SA 4\.0,[https://creativecommons\.org/licenses/by\-sa/4\.0/](https://creativecommons.org/licenses/by-sa/4.0/)\)\. The data was used unmodified\.

## Appendix CSupplementary videos

We host the supplementary videos demonstrating the rodent performing the various tasks described in this work on the project’s page \([https://aidansirbu\.com/SCAMPER\-site/](https://aidansirbu.com/SCAMPER-site/)\)\. Specific videos are also referenced throughout the text: “SuppVid<x\>” refers to the corresponding video on the website and the \.mp4 file in the supplementary video package\.

## Appendix DImplementation & compute

Implementation\.SCAMPER is implemented on top of the MuJoCo physics engine\[[43](https://arxiv.org/html/2610.27279#bib.bib43)\]\. We accelerate simulation using MuJoCo XLA \(MJX\)\[[44](https://arxiv.org/html/2610.27279#bib.bib44)\], a JAX\-based\[[45](https://arxiv.org/html/2610.27279#bib.bib45)\]reimplementation of MuJoCo, which enables GPU\-vectorized physics rollouts and domain randomization to be applied directly to the per\-environment parameters\. Environments are built on top of MuJoCo Playground’s\[[46](https://arxiv.org/html/2610.27279#bib.bib46)\]base class and wrappers, which expose MJX simulation through a JAX\-compatible interface and adapt our environments to the Brax\[[47](https://arxiv.org/html/2610.27279#bib.bib47)\]training API\. RL is performed with Brax, whose PPO implementation and distributed training utilities we extend to support our networks and distillation objectives\. Pretraining \(Stages 1\-3\) was performed using the JAX backend of MJX and task transfer was performed using the Warp backend of MJX\. MIMIC\-MJX baselines were implemented using the training setup and hyperparameters described in[Zhang et al\. \[6\]](https://arxiv.org/html/2610.27279#bib.bib6)\. The from\-scratch baseline uses position \(PD\) actuators, whereas all other models use torque actuators\. Torque actuators are highly unstable and difficult to learn from scratch, so we give the from\-scratch model PD actuators to give it a fair chance to compete\.

Compute\.All experiments were run on a single GPU with44CPU cores and4848GB of host RAM\. Pretraining used NVIDIA L40S \(48 GB\) GPUs: the Stage 1 expert imitator trains for3×1093\\times 10^\{9\}environment steps over40964096parallel envs in≈14\\approx 14h, the Stage 2 distillation of the encoder/decoder and motion prior trains for1×1091\\times 10^\{9\}steps over40964096envs in≈12\\approx 12h, and the Stage 3 behavior prior trains for2×1092\\times 10^\{9\}steps over40964096envs in≈30\\approx 30h, for a total of roughly5656L40S\-hours per end\-to\-end pretraining run\. The three task\-transfer benchmarks were trained with PPO under MuJoCo Warp on NVIDIA A100 \(80 GB\) GPUs, with four seeds per condition\. Walk\-then\-rear was run for the SCAMPER, MIMIC\-MJX, and from\-scratch models, with each job training for1×1091\\times 10^\{9\}steps over512512envs in≈11\\approx 11h \(1212jobs total,≈132\\approx 132A100\-hours\)\. Locomotion was run for the same three models swept over77target speeds, with each job training for4×1084\\times 10^\{8\}steps over10241024envs in≈10\\approx 10h \(8484jobs total,≈840\\approx 840A100\-hours\)\. Arena foraging was run for SCAMPER, a motion\-prior\-only variant, MIMIC\-MJX, and the from\-scratch model swept over77minimum spawn radii, with each job training for2×1092\\times 10^\{9\}steps over10241024envs in≈6\\approx 6h \(112112jobs total,≈672\\approx 672A100\-hours\)\. One full reproduction of the paper’s training pipeline therefore costs roughly5656L40S\-GPU\-hours and1,6501\{,\}650A100\-GPU\-hours\.

## Appendix ETasks

### E\.1Stage 1: Expert imitator

Task\.The expert imitatorπI​\(𝐚t\|𝐱t,𝐠t\)\\pi\_\{I\}\(\\mathbf\{a\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\)tracks 3D motion\-capture trajectories of freely behaving rats using the biomechanical rat model of[Merel et al\. \[4\]](https://arxiv.org/html/2610.27279#bib.bib4)\. We use a∼\\sim1 h subset of the open\-source rat mocap dataset released by[Aldarondo et al\. \[5\]](https://arxiv.org/html/2610.27279#bib.bib5)\(842 clips of 5 s mocap\), registered to the virtual rat via STAC\-MJX\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\]\. At the start of each episode we sample a clip uniformly from the training partition \(80%80\\%of the dataset; the remaining20%20\\%is held out for evaluation rollouts\) and a starting frame uniformly in\[0,50\]\[0,50\], with zero initial joint velocity\. Episodes last up to250250reference frames\. At each control step the agent observes the proprioception vector𝐱t\\mathbf\{x\}\_\{t\}and the egocentric goal𝐠t\\mathbf\{g\}\_\{t\}defined in Section[3\.1](https://arxiv.org/html/2610.27279#S3.SS1), where𝐠t\\mathbf\{g\}\_\{t\}is computed from a55\-frame reference horizon, and emits torques𝐚t∈ℝ38\\mathbf\{a\}\_\{t\}\\in\\mathbb\{R\}^\{38\}applied to the rat’s actuated joints\. Control runs at100100Hz on top of a500500Hz MuJoCo MJX physics simulation, with a5050Hz mocap reference\. The body model uses torque actuators and a0\.9×0\.9\\timesrescaling factor\.

Reward\.Each imitation tracking term takes the formri=wi​exp⁡\(−12​\(ei/σi\)2\)r\_\{i\}=w\_\{i\}\\exp\\\!\\big\(\-\\tfrac\{1\}\{2\}\(e\_\{i\}/\\sigma\_\{i\}\)^\{2\}\\big\), a Gaussian kernel of the per\-term erroreie\_\{i\}with characteristic scaleσi\\sigma\_\{i\}; values of\(wi,σi\)\(w\_\{i\},\\sigma\_\{i\}\)are tabulated in Table[2](https://arxiv.org/html/2610.27279#A5.T2)\. The four active tracking terms penalize errors in root position, root orientation \(quaternion geodesic\), joint angles, and egocentric Cartesian end\-effector positions\. Three regularization costs penalize quadratic control magnitude, quadratic control differences \(jerk\), and clipped torque–velocity product, and a torso\-zz\-range term adds an in\-range bonus that rewards keeping the torso height within the healthy range\[0\.0325,0\.5\]\[0\.0325,0\.5\]m\.

Termination\.An episode terminates early if \(i\) the root position deviates from the reference by more than0\.10\.1m, \(ii\) the root orientation deviates by more than60∘60^\{\\circ\}, \(iii\) the poseL2L\_\{2\}error exceeds4\.54\.5, or \(iv\) any state value becomesNaN\. Episodes also terminate at the end of the reference clip\.

Domain randomization\.To encourage robust control across variations in the biomechanical model, at the start of each episode we sample physics parameters uniformly within fixed ranges and hold them fixed for the duration of the episode\[[48](https://arxiv.org/html/2610.27279#bib.bib48)\]\. Randomized parameters include floor friction, static friction scale, joint armature scale, link mass scale, torso mass jitter, center\-of\-mass jitter, and an additive jitter on the initial joint configuration𝜽0\\boldsymbol\{\\theta\}\_\{0\}; full ranges are listed in Table[2](https://arxiv.org/html/2610.27279#A5.T2)\.

Network architecture\.The expert imitator is a multi\-layer perceptron with no latent bottleneck: the concatenation of𝐱t\\mathbf\{x\}\_\{t\}and𝐠t\\mathbf\{g\}\_\{t\}, normalized by their running mean and variance, is fed directly to a\[2048,1024,1024,512\]\[2048,1024,1024,512\]policy MLP that outputs the mean and log\-standard\-deviation of a tanh\-squashed diagonal Gaussian over the3838\-dimensional action, so that𝐚t∈\[−1,1\]38\\mathbf\{a\}\_\{t\}\\in\[\-1,1\]^\{38\}\. The critic is a separate\[2048,1024,1024,1024,1024,512\]\[2048,1024,1024,1024,1024,512\]MLP estimating the state\-value function\. Both networks use SiLU activations with LayerNorm between hidden layers, and are trained from random initialization using LeCun initialization with zero bias\.

Training\.We trainπI\\pi\_\{I\}with PPO\[[32](https://arxiv.org/html/2610.27279#bib.bib32)\]for3×1093\\times 10^\{9\}environment steps across4,0964\{,\}096parallel environments\. Each PPO update uses a2020\-step unroll, batch size1,0241\{,\}024, and44epochs of1616minibatches per batch, with Adam at learning rate10−410^\{\-4\}and gradient clipping at20\.020\.0\. We use discountγ=0\.95\\gamma=0\.95, PPO clip rangeε=0\.2\\varepsilon=0\.2, entropy coefficient10−210^\{\-2\}, value\-loss coefficient0\.250\.25, and reward scaling11\. Evaluation rollouts on the held\-out clip partition are performed every2\.5×1072\.5\\times 10^\{7\}environment steps with deterministic \(mean\) actions\. Training is implemented in JAX\[[45](https://arxiv.org/html/2610.27279#bib.bib45)\]with MuJoCo MJX\[[44](https://arxiv.org/html/2610.27279#bib.bib44)\]and runs on a single GPU; the full list of training hyperparameters is given in Table[2](https://arxiv.org/html/2610.27279#A5.T2)\.

### E\.2Stage 2: Online distillation

Task\.Online distillation reuses the imitation environment from Stage 1 \(App\.[E\.1](https://arxiv.org/html/2610.27279#A5.SS1)\): identical clip set and train/eval split, control rate, physics solver, reward terms, termination criteria, and domain randomization \(Table[2](https://arxiv.org/html/2610.27279#A5.T2)\)\. The Stage 1 expert imitatorπI\\pi\_\{I\}is loaded as a frozen teacher and queried for target actions𝐚t\\mathbf\{a\}\_\{t\}on every visited state, while a student latent skill controller—the encoderqϕ​\(𝐳t\|𝐱t,𝐠t\)q\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\), decoderpθ​\(𝐚t\|𝐱t,𝐳t\)p\_\{\\theta\}\(\\mathbf\{a\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{z\}\_\{t\}\), and proprioception\-conditioned motion priorpψ​\(𝐳t\|𝐱t\)p\_\{\\psi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)—is rolled out in simulation and trained jointly to match the teacher, using a similar DAgger\[[33](https://arxiv.org/html/2610.27279#bib.bib33)\]schedule as PULSE\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\]\. Episode terminations and rewards are inherited from the underlying environment but are not used as a learning signal: the student is supervised against the teacher’s actions, not the imitation reward\.

Loss\.The student minimizes the composite distillation objective

ℒdistill=ℒDAgger\+λ​ℒKL\+α​ℒAR,\\mathcal\{L\}\_\{\\mathrm\{distill\}\}=\\mathcal\{L\}\_\{\\mathrm\{DAgger\}\}\+\\lambda\\mathcal\{L\}\_\{\\mathrm\{KL\}\}\+\\alpha\\mathcal\{L\}\_\{\\mathrm\{AR\}\},whereℒKL=DKL\(qϕ\|\|pψ\)\\mathcal\{L\}\_\{\\mathrm\{KL\}\}=D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\|\|p\_\{\\psi\}\)regularizes the encoder toward the motion prior andℒAR=‖𝝁te−𝝁t−1e‖22\\mathcal\{L\}\_\{\\mathrm\{AR\}\}=\|\|\\boldsymbol\{\\mu\}\_\{t\}^\{e\}\-\\boldsymbol\{\\mu\}\_\{t\-1\}^\{e\}\|\|\_\{2\}^\{2\}promotes temporal smoothness in the encoder mean\. In practiceℒDAgger\\mathcal\{L\}\_\{\\mathrm\{DAgger\}\}is computed on the policy’s action\-distribution parameters rather than on sampled actions: both the teacher MLPπI\\pi\_\{I\}and the student decoderpθp\_\{\\theta\}output the mean and log\-standard\-deviation of a tanh\-squashed diagonal Gaussian over the action, and we take the MSE between these parameter vectors\. This reduces to the action\-matching formℒDAgger=‖𝐚t−𝐚^t‖22\\mathcal\{L\}\_\{\\mathrm\{DAgger\}\}=\|\|\\mathbf\{a\}\_\{t\}\-\\mathbf\{\\hat\{a\}\}\_\{t\}\|\|\_\{2\}^\{2\}given in the main text when both policies are evaluated deterministically \(i\.e\., at their distribution means\), and is a policy\-distribution distillation in the spirit of DAgger\[[33](https://arxiv.org/html/2610.27279#bib.bib33)\]more generally\. The KL coefficientλ\\lambdais annealed linearly from10−310^\{\-3\}to10−410^\{\-4\}over the first25%25\\%of training and held fixed thereafter; the AR\(1\) coefficient is fixed atα=5×10−3\\alpha=5\\times 10^\{\-3\}\. To prevent posterior collapse and runaway sampling noise, the encoder and prior log\-variances are clipped to\[−5,0\]\[\-5,0\]\(matching PULSE\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\]\)\.

Network architecture\.All three networks are MLPs with SiLU activations and LayerNorm between hidden layers \(matching the Stage 1 policy\)\. The encoder maps the concatenation of𝐱t\\mathbf\{x\}\_\{t\}and𝐠t\\mathbf\{g\}\_\{t\}through hidden layers\[2048,1024,1024\]\[2048,1024,1024\]to per\-dimension means and log\-variances of a1616\-dimensional Gaussian latent𝐳t\\mathbf\{z\}\_\{t\}; we sample𝐳t\\mathbf\{z\}\_\{t\}via the standard reparameterization trick\. The decoder maps the concatenation of𝐳t\\mathbf\{z\}\_\{t\}and𝐱t\\mathbf\{x\}\_\{t\}through\[1024,1024,512\]\[1024,1024,512\]to an action prediction𝐚^t\\hat\{\\mathbf\{a\}\}\_\{t\}\. The proprioception\-conditioned motion prior maps𝐱t\\mathbf\{x\}\_\{t\}through\[1536,1024,512\]\[1536,1024,512\]to the mean and log\-variance ofpψ​\(𝐳t\|𝐱t\)p\_\{\\psi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)\. Encoder, decoder, and motion prior are randomly initialized using LeCun initialization with zero bias and trained end\-to\-end; the teacherπI\\pi\_\{I\}remains frozen throughout\.

Training\.We collect4,0964\{,\}096parallel student rollouts in MuJoCo MJX with the same2020\-step unroll length used in Stage 1, train for10910^\{9\}environment steps with batch size1,0241\{,\}024split across1616minibatches and44epochs per batch, and update\(ϕ,θ,ψ\)\(\\phi,\\theta,\\psi\)jointly with Adam at learning rate10−410^\{\-4\}and global gradient\-norm clipping at10\.010\.0\. Observations are normalized by their running mean and variance, with statistics inherited from the Stage 1 normalizer to ensure consistency with the frozen teacher\. Evaluation rollouts are run every2×1072\\times 10^\{7\}environment steps with the deterministic decoder mean\. The complete list of distillation hyperparameters is given in Table[3](https://arxiv.org/html/2610.27279#A5.T3)\.

### E\.3Stage 3: Behavior prior distillation

Task\.Behavior prior distillation reuses the imitation environment from Stage 1 \(App\.[E\.1](https://arxiv.org/html/2610.27279#A5.SS1)\) for rollout collection, with domain randomization disabled\. The Stage 2 latent skill controller—encoderqϕ​\(𝐳t\|𝐱t,𝐠t\)q\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\), decoderpθ​\(𝐚t\|𝐱t,𝐳t\)p\_\{\\theta\}\(\\mathbf\{a\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{z\}\_\{t\}\), and motion priorpψ​\(𝐳t\|𝐱t\)p\_\{\\psi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)—is loaded frozen and used to drive simulation: at each step we feed the encoder’s mean𝝁te\\boldsymbol\{\\mu\}^\{e\}\_\{t\}as the latent into the frozen decoder and use the resulting deterministic action mean to advance the environment\. The trainable behavior priorpφ​\(𝐳t\|𝐱t\)p\_\{\\varphi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)is optimized only on the proprioceptive states visited along these trajectories; loss evaluation queries the full encoder distribution\(𝝁te,𝝈te\)\(\\boldsymbol\{\\mu\}^\{e\}\_\{t\},\\boldsymbol\{\\sigma\}^\{e\}\_\{t\}\)at each visited state, not a sample\.

Loss\.With the encoderqϕq\_\{\\phi\}held fixed, the behavior prior is trained by minimizing the expected KL divergence

ℒbehavior=DKL\(qϕ\(𝐳t\|𝐱t,𝐠t\)∥pφ\(𝐳t\|𝐱t\)\),\\mathcal\{L\}\_\{\\mathrm\{behavior\}\}=D\_\{\\mathrm\{KL\}\}\\big\(q\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{g\}\_\{t\}\)\\,\\\|\\,p\_\{\\varphi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)\\big\),which by Proposition[2](https://arxiv.org/html/2610.27279#Thmproposition2)is equivalent to matching the marginal posteriormϕ​\(𝐳t\|𝐱t\)=𝔼p⁡\(𝐠t\|𝐱t\)​\[qϕ​\(𝐳t\|𝐠t,𝐱t\)\]m\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)=\\mathbb\{E\}\_\{p\(\\mathbf\{g\}\_\{t\}\|\\mathbf\{x\}\_\{t\}\)\}\[q\_\{\\phi\}\(\\mathbf\{z\}\_\{t\}\|\\mathbf\{g\}\_\{t\},\\mathbf\{x\}\_\{t\}\)\]\. Crucially, the encoder is not modified at this stage; it provides a fixed target distribution, so the behavior prior absorbs the goal\-conditional variation inqϕq\_\{\\phi\}rather than collapsing onto local motion continuation as the motion prior does \(Section[3\.3](https://arxiv.org/html/2610.27279#S3.SS3)\)\.

Network architecture\.The behavior prior is an MLP with SiLU activations and LayerNorm between hidden layers, mirroring the motion prior\. It maps𝐱t\\mathbf\{x\}\_\{t\}through hidden sizes\[2048,2048,1024,512\]\[2048,2048,1024,512\]to the mean and log\-variance of a1616\-dimensional Gaussian over𝐳t\\mathbf\{z\}\_\{t\}, matching the latent dimensionality of the frozen encoder\. The behavior prior is the only trainable network in this stage; encoder, decoder, and motion prior all remain frozen\. The behavior prior is initialized using LeCun initialization with zero bias\.

Training\.We collect4,0964\{,\}096parallel rollouts in MuJoCo MJX with the same2020\-step unroll length used in earlier stages, train for2×1092\\times 10^\{9\}environment steps with batch size1,0241\{,\}024split across1616minibatches and44epochs per batch, and updateφ\\varphiwith Adam at learning rate10−410^\{\-4\}and global gradient\-norm clipping at10\.010\.0\. Observation statistics are inherited from the Stage 2 normalizer and held fixed, so thatpφp\_\{\\varphi\}sees the same input distribution as the frozen encoder\. Evaluation rollouts are run every2×1072\\times 10^\{7\}environment steps using the deterministic decoder mean and a deterministic behavior\-prior mean \(with stochastic\-sampling evaluations reported separately in the experiments\)\. The complete list of behavior\-prior hyperparameters is given in Table[4](https://arxiv.org/html/2610.27279#A5.T4)\.

### E\.4Stage 4 \(transfer learning\): Arena foraging

Task\.Stage 4 fine\-tunes a task\-specific policy on top of the frozen Stage 3 controller\. The decoderpθp\_\{\\theta\}, motion priorpψp\_\{\\psi\}, and behavior priorpφp\_\{\\varphi\}are loaded frozen from the Stage 3 checkpoint \(Table[4](https://arxiv.org/html/2610.27279#A5.T4)\); only the task policyπT\(𝜹t,βt\|𝐱t,𝐨t\)\\pi\_\{T\}\(\\boldsymbol\{\\delta\}\_\{t\},\\beta\_\{t\}\|\\mathbf\{x\}\_\{t\},\\mathbf\{o\}\_\{t\}\)and its critic are trained\. In the arena foraging task, the rat is placed at the center of a flat circular arena \(Fig\.[11](https://arxiv.org/html/2610.27279#A8.F11)\) and must navigate to and collectN=10N=10treats \(blue spheres of radius0\.060\.06m at height0\.030\.03m\)\. At the start of each episode the rat’s initial yaw is sampled uniformly in\[0,2​π\)\[0,2\\pi\), and theNNtreats are placed one per angular sector with sub\-sector jitter, at uniformly random radii within an annulus of inner radius of variable size depending on the trial and outer radiusrmax=2\.0r\_\{\\max\}=2\.0m\. Each treat emits an isotropic Gaussian scent field with bandwidthσ=0\.2\\sigma=0\.2m\. In addition to the proprioception vector𝐱t\\mathbf\{x\}\_\{t\}from Section[3\.1](https://arxiv.org/html/2610.27279#S3.SS1), the agent observes a task\-specific𝐨t\\mathbf\{o\}\_\{t\}comprising \(i\) the scalar total scent at the rat’s skull \(sum of Gaussians from currently active treats\) and \(ii\) the egocentric gradient∇st∈ℝ3\\nabla\\\!s\_\{t\}\\in\\mathbb\{R\}^\{3\}of the same scent field, which points toward the scent\-weighted treat center\.

Reward\.The task uses a sparse reward: collecting a treat \(skull entering its collision sphere\) yields a large, one\-time reward and the treat with its scent contribution is removed for the remainder of the episode\.

Termination\.Each episode runs for up to2,0002\{,\}000control steps \(2020s at the100100Hz control rate inherited from Stage 1\), or terminates early once allNNtreats have been collected\.

Policy gating\.The task policy outputs a1616\-dimensional residual𝜹t\\boldsymbol\{\\delta\}\_\{t\}to the motion prior’s mean and a scalar gating logitβt\\beta\_\{t\}\. The binary gatebtb\_\{t\}is thresholded with bias0\.50\.5and hysteresis margin0\.20\.2: switchingbtb\_\{t\}to11requiresβt\>0\.7\\beta\_\{t\}\>0\.7, and switching to00requiresβt<0\.3\\beta\_\{t\}<0\.3\. Whenbt=1b\_\{t\}=1, the latent is𝐳t=𝝁tm\+𝜹t\\mathbf\{z\}\_\{t\}=\\boldsymbol\{\\mu\}\_\{t\}^\{m\}\+\\boldsymbol\{\\delta\}\_\{t\}\(motion\-prior residual control\)\. Whenbt=0b\_\{t\}=0, the latent is sampled stochastically from the behavior prior,𝐳t=𝝁tb\+ϵ​𝝈\\mathbf\{z\}\_\{t\}=\\boldsymbol\{\\mu\}\_\{t\}^\{b\}\+\\epsilon\\boldsymbol\{\\sigma\}withϵ∼𝒩⁡\(0,1\)\\epsilon\\\!\\sim\\\!\\mathcal\{N\}\(0,1\)and a fixed log\-variance of−2\.25\-2\.25\(i\.e\.,σ=0\.32\\sigma=0\.32\), which yields random motions that fall within the range of the natural behavioral diversity of the training data \(Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)b\)\. The latent is passed through the frozen decoderpθp\_\{\\theta\}to produce the action torques\.

Network architecture\.The task policy is an MLP with hidden sizes\[2048,1024,512\]\[2048,1024,512\]that maps the concatenation of𝐱t\\mathbf\{x\}\_\{t\}and𝐨t\\mathbf\{o\}\_\{t\}\(normalized by running mean and variance\) to the mean and log\-standard\-deviation of a tanh\-squashed diagonal Gaussian over the joint1717\-dimensional output\(𝜹t,βt\)\(\\boldsymbol\{\\delta\}\_\{t\},\\beta\_\{t\}\), so that𝜹t∈\[−1,1\]16\\boldsymbol\{\\delta\}\_\{t\}\\in\[\-1,1\]^\{16\}andβt∈\[−1,1\]\\beta\_\{t\}\\in\[\-1,1\]\. The critic is a separate\[2048,1024,1024,512\]\[2048,1024,1024,512\]MLP estimating the state\-value function\. Both networks use SiLU activations with LayerNorm between hidden layers and are randomly initialized using LeCun initialization with zero bias\. The decoder, motion prior, and behavior prior all remain frozen throughout this stage\.

Training\.We trainπT\\pi\_\{T\}with PPO\[[32](https://arxiv.org/html/2610.27279#bib.bib32)\]for2×1092\\times 10^\{9\}environment steps across1,0241\{,\}024parallel environments\. Each PPO update uses a2020\-step unroll, batch size1,0241\{,\}024, and44epochs of1616minibatches per batch, with Adam at learning rate10−410^\{\-4\}and gradient clipping at maximum norm50\.050\.0\. We use discountγ=0\.99\\gamma=0\.99, GAEλ=0\.95\\lambda=0\.95, PPO clip rangeε=0\.3\\varepsilon=0\.3, entropy coefficient10−210^\{\-2\}, value\-loss coefficient0\.50\.5, and reward scaling1\.01\.0\. Evaluation rollouts are performed every1\.5×1071\.5\\times 10^\{7\}environment steps\. The complete list of hyperparameters is given in Table[5](https://arxiv.org/html/2610.27279#A5.T5)\.

### E\.5Locomotion

Task\.The locomotion task assesses how effectively the motion priorpψp\_\{\\psi\}supports goal\-directed control via residual fine\-tuning, without recourse to the behavior prior\. The motion priorpψp\_\{\\psi\}and decoderpθp\_\{\\theta\}are loaded frozen from the Stage 2 checkpoint \(Table[3](https://arxiv.org/html/2610.27279#A5.T3)\); only the task policy and critic are trained\. We do not use the behavior prior in this task: we usebt=1b\_\{t\}=1at every step, so that the latent is always𝐳t=𝝁tm\+𝜹t\\mathbf\{z\}\_\{t\}=\\boldsymbol\{\\mu\}\_\{t\}^\{m\}\+\\boldsymbol\{\\delta\}\_\{t\}, isolating the quality of residual control of the motion prior\. At the start of each episode the rat is placed at the world origin in a neutral standing pose facing the world\+x\+xdirection \(identity body quaternion,z=0z=0\), on a flat arena\. The task is to maintain a fixed target forward speedv⋆v^\{\\star\}along the world\+x\+xaxis\. In addition to the proprioception vector𝐱t\\mathbf\{x\}\_\{t\}from Section[3\.1](https://arxiv.org/html/2610.27279#S3.SS1), the agent receives a task observation𝐨t\\mathbf\{o\}\_\{t\}comprising the previous action, and the rat’s kinematic and touch sensors\. The target speedv⋆v^\{\\star\}itself is a fixed task parameter and is not part of the observation\.

Reward\.The reward at each step is a piecewise\-linear forward\-velocity tracking signal,

rt=max⁡\(0,1−\|vtx−v⋆\|/v⋆\),r\_\{t\}=\\max\\\!\\Big\(0,\\;1\-\|v^\{x\}\_\{t\}\-v^\{\\star\}\|/v^\{\\star\}\\Big\),wherevtxv^\{x\}\_\{t\}is the world\-frame\+x\+xcomponent of the torso’s linear velocity \(subtree center\-of\-mass velocity of the torso body\)\. The reward equals11whenvtx=v⋆v^\{x\}\_\{t\}=v^\{\\star\}and decays linearly to00atvtx=0v^\{x\}\_\{t\}=0and atvtx=2​v⋆v^\{x\}\_\{t\}=2v^\{\\star\}\.

Termination\.An episode terminates early if \(i\) the rat has*fallen*—defined as torsozzbelow0\.030\.03m or torso tilt from vertical exceeding60∘60^\{\\circ\}—or \(ii\) any state value becomesNaN\. Episodes are otherwise truncated at a fixed length of1,0001\{,\}000control steps \(1010s at the100100Hz control rate inherited from Stage 1\)\.

Network architecture\.The task policy is an MLP with hidden sizes\[2048,1024,512\]\[2048,1024,512\]that maps the concatenation of𝐱t\\mathbf\{x\}\_\{t\}and𝐨t\\mathbf\{o\}\_\{t\}\(normalized by running mean and variance\) to the mean and log\-standard\-deviation of a tanh\-squashed diagonal Gaussian over the residual, so that𝜹t∈\[−1,1\]16\\boldsymbol\{\\delta\}\_\{t\}\\in\[\-1,1\]^\{16\}\. The critic is a separate\[2048,1024,1024,512\]\[2048,1024,1024,512\]MLP estimating the state\-value function\. Both networks use SiLU activations with LayerNorm between hidden layers and are randomly initialized using LeCun initialization with zero bias\. The motion priorpψp\_\{\\psi\}and decoderpθp\_\{\\theta\}remain frozen throughout this task; the encoderqϕq\_\{\\phi\}is not used\.

Training\.We train the task policy with PPO\[[32](https://arxiv.org/html/2610.27279#bib.bib32)\]for4×1084\\times 10^\{8\}environment steps across1,0241\{,\}024parallel environments\. Each PPO update uses a2020\-step unroll, batch size1,0241\{,\}024, and44epochs of1616minibatches per batch, with Adam at learning rate10−410^\{\-4\}and gradient clipping at maximum norm50\.050\.0\. We use discountγ=0\.99\\gamma=0\.99, GAEλ=0\.95\\lambda=0\.95, PPO clip rangeε=0\.3\\varepsilon=0\.3, entropy coefficient10−210^\{\-2\}, value\-loss coefficient0\.50\.5, and reward scaling1\.01\.0\. Evaluation rollouts are performed every1\.5×1071\.5\\times 10^\{7\}environment steps\. The complete list of hyperparameters is given in Table[6](https://arxiv.org/html/2610.27279#A5.T6)\.

### E\.6Rearing

Task\.The walk\-then\-rear task tests whether residual control of the motion prior can drive a navigate\-and\-rear sequence terminated by a sparse success bonus\. As in the locomotion task, the motion priorpψp\_\{\\psi\}and decoderpθp\_\{\\theta\}are loaded frozen from the Stage 2 checkpoint \(Table[3](https://arxiv.org/html/2610.27279#A5.T3)\); only the task policy and critic are trained, and we usebt=1b\_\{t\}=1at every step so that the latent is always𝐳t=𝝁tm\+𝜹t\\mathbf\{z\}\_\{t\}=\\boldsymbol\{\\mu\}\_\{t\}^\{m\}\+\\boldsymbol\{\\delta\}\_\{t\}\. At the start of each episode the rat is placed at the world origin in a neutral standing pose facing the world\+x\+xdirection \(identity body quaternion,z=0z=0\), on a flat arena; a translucent target box is rendered at the goal location for visualization\. The agent must walk to the horizontal target\(x⋆,y⋆\)=\(1\.0,0\.0\)\(x^\{\\star\},y^\{\\star\}\)=\(1\.0,\\,0\.0\)m and rear up so that its skull reaches a target heightz⋆=0\.2z^\{\\star\}=0\.2m\. A target pose is achieved when the skull is simultaneously within0\.010\.01m of\(x⋆,y⋆\)\(x^\{\\star\},y^\{\\star\}\)horizontally and within0\.020\.02m ofz⋆z^\{\\star\}vertically; success requires holding the target pose continuously for1\.01\.0s \(100100control steps at100100Hz\)\. In addition to the proprioception vector𝐱t\\mathbf\{x\}\_\{t\}from Section[3\.1](https://arxiv.org/html/2610.27279#S3.SS1), the agent receives a task observation𝐨t\\mathbf\{o\}\_\{t\}comprising the previous action, the rat’s kinematic and touch sensors, the egocentric position of the world origin in the torso frame, and the egocentric position of the target in the torso frame\.

Reward\.The reward at each step combines a dense piecewise\-linear shaping term and a sparse terminal bonus\. Lettingdt=‖𝐩tskull−𝐩⋆‖2d\_\{t\}=\\\|\\mathbf\{p\}^\{\\text\{skull\}\}\_\{t\}\-\\mathbf\{p\}^\{\\star\}\\\|\_\{2\}denote the Euclidean distance from the skull to the target, the dense term is

rtdist=max⁡\(0,1−dt/m\),r^\{\\text\{dist\}\}\_\{t\}=\\max\\\!\\Big\(0,\\;1\-d\_\{t\}/m\\Big\),with shaping marginm=1\.0m=1\.0m, so it equals11when the skull is at the target and decays linearly to00atdt=md\_\{t\}=m\. The sparse bonusrtsucc=500r^\{\\text\{succ\}\}\_\{t\}=500fires on the single step at which the running hold counter first reaches100100steps, and is00otherwise\.

Termination\.An episode terminates if \(i\) the hold counter reaches100100steps \(success\), \(ii\) the rat has*fallen*—defined as torsozzbelow0\.020\.02m or torso tilt from vertical exceeding80∘80^\{\\circ\}—or \(iii\) any state value becomesNaN\. Episodes are otherwise truncated at a fixed length of500500control steps \(55s at the100100Hz control rate inherited from Stage 1\)\.

Network architecture\.The task policy is an MLP with hidden sizes\[2048,1024,512\]\[2048,1024,512\]that maps the concatenation of𝐱t\\mathbf\{x\}\_\{t\}and𝐨t\\mathbf\{o\}\_\{t\}\(normalized by running mean and variance\) to the mean and log\-standard\-deviation of a tanh\-squashed diagonal Gaussian over the residual, so that𝜹t∈\[−1,1\]16\\boldsymbol\{\\delta\}\_\{t\}\\in\[\-1,1\]^\{16\}\. The critic is a separate\[2048,1024,1024,512\]\[2048,1024,1024,512\]MLP estimating the state\-value function\. Both networks use SiLU activations with LayerNorm between hidden layers and are randomly initialized using LeCun initialization with zero bias\. The motion priorpψp\_\{\\psi\}and decoderpθp\_\{\\theta\}remain frozen throughout this task; the encoderqϕq\_\{\\phi\}is not used\.

Training\.We train the task policy with PPO\[[32](https://arxiv.org/html/2610.27279#bib.bib32)\]for6×1086\\times 10^\{8\}environment steps across1,0241\{,\}024parallel environments\. Each PPO update uses a2020\-step unroll, batch size1,0241\{,\}024, and44epochs of1616minibatches per batch, with Adam at learning rate5×10−55\\times 10^\{\-5\}and gradient clipping at maximum norm50\.050\.0\. We use discountγ=0\.99\\gamma=0\.99, GAEλ=0\.95\\lambda=0\.95, PPO clip rangeε=0\.3\\varepsilon=0\.3, entropy coefficient10−210^\{\-2\}, value\-loss coefficient0\.50\.5, and reward scaling1\.01\.0\. Evaluation rollouts are performed every1\.5×1071\.5\\times 10^\{7\}environment steps\. The complete list of hyperparameters is given in Table[7](https://arxiv.org/html/2610.27279#A5.T7)\.

### E\.7Hyperparameters

Our hyperparameter choices draw on prior work where applicable, with task\-specific tuning otherwise\. PPO hyperparameters follow Brax’s defaults\[[47](https://arxiv.org/html/2610.27279#bib.bib47)\]with minor adjustments to maximize task reward\. Physics simulation and virtual rodent parameters are taken from MIMIC\-MJX\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\], and online distillation hyperparameters follow PULSE\[[13](https://arxiv.org/html/2610.27279#bib.bib13)\]with small modifications identified via constrained grid search to better suit the virtual rodent setting\. All remaining hyperparameters were selected through grid search\.

Table 2:Hyperparameters used for Stage 1 \(train expert imitator\) of the SCAMPER training pipeline\.GroupHyperparameterValueEnvironmentSolverNewtonSolver iterations5Line\-search iterations5Max contacts \(naconmaxn\_\{\\text\{aconmax\}\}\)256Max constraint rows \(njmaxn\_\{\\text\{jmax\}\}\)128Simulation timestep2×10−32\\times 10^\{\-3\}s \(500 Hz\)Control timestep1×10−21\\times 10^\{\-2\}s \(100 Hz\)Mocap rate50 HzClip length250 framesReference horizon5 framesEpisode start\-frame range\[0,50\]\[0,50\]Initial joint velocityzerosTrain / eval clip split0\.8 / 0\.2Domain randomizationFloor friction𝒰⁡\(0\.4,1\.0\)\\mathcal\{U\}\(0\.4,1\.0\)Static friction scale𝒰⁡\(0\.9,1\.1\)\\mathcal\{U\}\(0\.9,1\.1\)Armature scale𝒰⁡\(1\.0,1\.05\)\\mathcal\{U\}\(1\.0,1\.05\)Center\-of\-mass jitter \(m\)𝒰⁡\(−0\.05,0\.05\)\\mathcal\{U\}\(\-0\.05,0\.05\)Link mass scale𝒰⁡\(0\.9,1\.1\)\\mathcal\{U\}\(0\.9,1\.1\)Torso mass jitter𝒰⁡\(−0\.02,0\.02\)\\mathcal\{U\}\(\-0\.02,0\.02\)Initialqposq\_\{\\text\{pos\}\}jitter𝒰⁡\(−0\.05,0\.05\)\\mathcal\{U\}\(\-0\.05,0\.05\)Reward weights \(ww, scale\)Root positionw=1\.0w\{=\}1\.0,σ=0\.035\\sigma\{=\}0\.035Root orientation \(quaternion\)w=1\.0w\{=\}1\.0,σ=20\.0\\sigma\{=\}20\.0Joint anglesw=1\.0w\{=\}1\.0,σ=1\.41\\sigma\{=\}1\.41Joint velocitiesw=0\.0w\{=\}0\.0,σ=1\.0\\sigma\{=\}1\.0Body positionsw=0\.0w\{=\}0\.0,σ=0\.25\\sigma\{=\}0\.25End\-effectorsw=1\.0w\{=\}1\.0,σ=0\.03\\sigma\{=\}0\.03Healthy torsozz\-range \(m\)\[0\.0325,0\.5\]\[0\.0325,0\.5\],w=1\.0w\{=\}1\.0Control costw=0\.02w\{=\}0\.02Control\-diff costw=0\.02w\{=\}0\.02Energy cost \(clipped at 50\)w=0\.01w\{=\}0\.01TerminationRoot\-position deviation\>0\.1\>0\.1mRoot\-orientation deviation\>60∘\>60^\{\\circ\}Pose error \(L2\)\>4\.5\>4\.5Network architecturePolicy architectureMLP \(no latent bottleneck\)Policy hidden sizes\[2048,1024,1024,512\]\[2048,1024,1024,512\]Critic hidden sizes\[2048,1024,1024,1024,1024,512\]\[2048,1024,1024,1024,1024,512\]Observation normalizationrunning mean/stdPPO trainingParallel environments4,096Total environment steps3×1093\\times 10^\{9\}Batch size1,024Minibatches per epoch16Updates per batch4Learning rate \(Adam\)1×10−41\\times 10^\{\-4\}Gradient clip threshold20\.0PPO clip rangeε\\varepsilon0\.2Discount factorγ\\gamma0\.95GAE / unroll length20Action repeat1Entropy coefficient1×10−21\\times 10^\{\-2\}Value\-loss coefficient0\.25Reward scaling1Evaluation / reset interval2\.5×1072\.5\\times 10^\{7\}steps

Table 3:Hyperparameters used for Stage 2 \(online distillation\) of the SCAMPER training pipeline\. Environment, termination, and domain\-randomization settings are inherited from Stage 1 \(Table[2](https://arxiv.org/html/2610.27279#A5.T2)\)\.GroupHyperparameterValueDistillation lossAR\(1\) smoothness weightα\\alpha5×10−35\\times 10^\{\-3\}Gradient clip threshold \(global norm\)10\.010\.0Encoder / prior logvar clamp\[−5,0\]\[\-5,0\]KL scheduleSchedule typelinear ramp, then holdStart weightλ0\\lambda\_\{0\}1×10−31\\times 10^\{\-3\}End weightλ1\\lambda\_\{1\}1×10−41\\times 10^\{\-4\}Ramp range \(fraction of training\)\[0%,25%\]\[0\\%,\\,25\\%\]Hard\-negative miningReweight strengthαHN\\alpha\_\{\\text\{HN\}\}0\.10\.1Max sample weightwmaxw\_\{\\text\{max\}\}5\.05\.0Network architectureLatent size1616Encoder hidden sizes\[2048,1024,1024\]\[2048,1024,1024\]Decoder hidden sizes\[1024,1024,512\]\[1024,1024,512\]Motion prior hidden sizes\[1536,1024,512\]\[1536,1024,512\]TrainingParallel environments4,0964\{,\}096Total environment steps1×1091\\times 10^\{9\}Batch size1,0241\{,\}024Minibatches per epoch1616Updates per batch44Learning rate \(Adam\)1×10−41\\times 10^\{\-4\}Unroll length2020Action repeat11Train / eval clip split0\.8/0\.20\.8/0\.2Evaluation / reset interval2×1072\\times 10^\{7\}steps

Table 4:Hyperparameters used for Stage 3 \(behavior prior distillation\) of the SCAMPER training pipeline\. Environment settings are inherited from Stage 1 \(Table[2](https://arxiv.org/html/2610.27279#A5.T2)\); the encoder, decoder, and motion prior are loaded frozen from Stage 2 \(Table[3](https://arxiv.org/html/2610.27279#A5.T3)\)\. Only the behavior prior is updated\.Table 5:Hyperparameters used to train SCAMPER on the arena foraging task \(Stage 4: transfer learning\)\. The decoder, motion prior, and behavior prior are loaded frozen from the Stage 3 checkpoint \(Table[4](https://arxiv.org/html/2610.27279#A5.T4)\); only the task policy and critic are trained at this stage\.GroupHyperparameterValueTask / environmentEpisode length2,0002\{,\}000control stepsEvaluation interval1\.5×1071\.5\\times 10^\{7\}env stepsNumber of treats per episode1010Treat sphere radius0\.060\.06mTreat height0\.030\.03mMinimum spawn radiusVariableMaximum spawn radius2\.02\.0mScent kernelσ\\sigma0\.20\.2mTreat\-collection reward500\.0500\.0Behavior–motion prior gatingGate thresholdbbiasb\_\{\\text\{bias\}\}0\.50\.5Gate hysteresis margin0\.20\.2Behavior\-prior fixed log\-variance−2\.25\-2\.25Network architectureTask policy hidden sizes\[2048,1024,512\]\[2048,1024,512\]Critic hidden sizes\[2048,1024,1024,512\]\[2048,1024,1024,512\]PPO trainingParallel environments1,0241\{,\}024Total environment steps2×1092\\times 10^\{9\}Batch size1,0241\{,\}024Minibatches per epoch1616Updates per batch44Learning rate \(Adam\)1×10−41\\times 10^\{\-4\}Gradient clip threshold \(max norm\)50\.050\.0PPO clip rangeε\\varepsilon0\.30\.3Discount factorγ\\gamma0\.990\.99GAEλ\\lambda0\.950\.95Entropy coefficient1×10−21\\times 10^\{\-2\}Value\-loss coefficient0\.50\.5Reward scaling1\.01\.0Unroll length2020Action repeat11

Table 6:Hyperparameters used to train SCAMPER on the locomotion task\. The motion priorpψp\_\{\\psi\}and decoderpθp\_\{\\theta\}are loaded frozen from the Stage 2 checkpoint \(Table[3](https://arxiv.org/html/2610.27279#A5.T3)\); only the task policy and critic are trained at this stage\. The policy outputs a residual𝜹t\\boldsymbol\{\\delta\}\_\{t\}added to the \(deterministic\) motion\-prior mean\.GroupHyperparameterValueTask / environmentEpisode length1,0001\{,\}000control stepsEvaluation interval1\.5×1071\.5\\times 10^\{7\}env stepsTarget forward speedVariableForward\-velocity reward weight1\.01\.0Network architectureTask policy hidden sizes\[2048,1024,512\]\[2048,1024,512\]Critic hidden sizes\[2048,1024,1024,512\]\[2048,1024,1024,512\]PPO trainingParallel environments1,0241\{,\}024Total environment steps4×1084\\times 10^\{8\}Batch size1,0241\{,\}024Minibatches per epoch1616Updates per batch44Learning rate \(Adam\)1×10−41\\times 10^\{\-4\}Gradient clip threshold \(max norm\)50\.050\.0PPO clip rangeε\\varepsilon0\.30\.3Discount factorγ\\gamma0\.990\.99GAEλ\\lambda0\.950\.95Entropy coefficient1×10−21\\times 10^\{\-2\}Value\-loss coefficient0\.50\.5Reward scaling1\.01\.0Unroll length2020Action repeat11

Table 7:Hyperparameters used to train SCAMPER on the walk\-rear task\. The motion priorpψp\_\{\\psi\}and decoderpθp\_\{\\theta\}are loaded frozen from the Stage 2 checkpoint \(Table[3](https://arxiv.org/html/2610.27279#A5.T3)\); only the task policy and critic are trained\. The policy outputs a residual𝜹t\\boldsymbol\{\\delta\}\_\{t\}added to the \(deterministic\) motion\-prior mean\.GroupHyperparameterValueTask / environmentEpisode length500500control stepsEvaluation interval1\.5×1071\.5\\times 10^\{7\}env stepsTarget position\(x⋆,y⋆,z⋆\)\(x^\{\\star\},y^\{\\star\},z^\{\\star\}\)\(1\.0,0\.0,0\.2\)\(1\.0,\\,0\.0,\\,0\.2\)mx​yxy\-tolerance for success0\.010\.01mzz\-tolerance for success0\.020\.02mRequired hold duration1\.01\.0sDistance\-shaping margin1\.01\.0mDistance\-to\-target reward weight1\.01\.0Walk\-rear success reward500\.0500\.0Network architectureTask policy hidden sizes\[2048,1024,512\]\[2048,1024,512\]Critic hidden sizes\[2048,1024,1024,512\]\[2048,1024,1024,512\]PPO trainingParallel environments1,0241\{,\}024Total environment steps6×1086\\times 10^\{8\}Batch size1,0241\{,\}024Minibatches per epoch1616Updates per batch44Learning rate \(Adam\)5×10−55\\times 10^\{\-5\}Gradient clip threshold \(max norm\)50\.050\.0PPO clip rangeε\\varepsilon0\.30\.3Discount factorγ\\gamma0\.990\.99GAEλ\\lambda0\.950\.95Entropy coefficient1×10−21\\times 10^\{\-2\}Value\-loss coefficient0\.50\.5Reward scaling1\.01\.0Unroll length2020Action repeat11

## Appendix FAnalysis of randomly generated behavior

### F\.1Rollout collection protocol

All quantitative comparisons in this section \(KID, Vendi score, root\-height ECDF, spatial occupancy\) are computed on rollouts collected with a single shared protocol so that the only difference between conditions is the controller that produced the trajectory\.

For every condition, we start from the same set of initial states: the first frame of each of the842842reference clips, with the rat placed in the corresponding generalized pose and zero joint velocity\. This matches the initialization used during imitation training and keeps the starting distribution of body configurations identical across conditions\. Each rollout is then advanced in MuJoCo MJX under the chosen controller for the same horizon as the reference clip \(∼\\sim55s of simulated time, with control at100100Hz on top of a500500Hz physics step\); the trajectory is driven purely by stochastic sampling of the controllers\. At each step we log the rat’s generalized coordinates𝐪∈ℝ74\\mathbf\{q\}\\in\\mathbb\{R\}^\{74\}and the latent vector passed to the decoder\. Clips are processed in parallel batches on a single GPU, and one randomization key is split per condition so that stochastic samples are reproducible and independent across conditions\. We perform the rollouts using both the behavior prior and the motion prior and compare to the imitation rollouts as the ground truth\. We test across both the deterministic sampling condition \(no noise;σ=0\\sigma=0\) and an array of different sampling noise levelslogvar=\{−4,−3,−2,−1,−0\.5,0\}\\text\{logvar\}=\\\{\-4,\-3,\-2,\-1,\-0\.5,0\\\}\(σ≈\{0\.14,0\.22,0\.37,0\.61,0\.78,1\.00\}\\sigma\\approx\\\{0\.14,0\.22,0\.37,0\.61,0\.78,1\.00\\\}\)\.

### F\.2Vendi score

##### What the Vendi score measures\.

The Vendi score\[[34](https://arxiv.org/html/2610.27279#bib.bib34)\]quantifies the effective number of distinct modes in a sample under a user\-specified similarity kernel\. It is defined as the exponential of the von Neumann \(Shannon\) entropy of the eigenvalues of the normalized similarity matrix and, in contrast to a clustering\-based count, requires no reference distribution, no labels, and no choice of cluster number\. It reduces to the number of items when they are mutually orthogonal and to11when they are all identical\. Higher Vendi score indicates greater behavioral diversity\.

##### Egocentric kinematic features\.

Unlike for KID, where the feature extractor is a learned VAE, we measure diversity directly in an egocentric kinematic feature space so that the score is interpretable in terms of physically meaningful movement variation rather than the geometry of a learned latent space\. From each rollout’s generalized coordinates𝐪∈ℝ74\\mathbf\{q\}\\in\\mathbb\{R\}^\{74\}, we compute, per frame, the root height, root roll and pitch, body\-frame linear velocity, yaw rate, and the joint angles —7474dims/frame\. The body\-frame velocities are intrinsically heading\-invariant and the planar root translation is dropped, so the representation is invariant to the absolute location of the rat in the arena\. Frames are then stacked into sliding windows of length3030\(0\.60\.6s at the5050Hz control rate\) with a stride of1515, yielding2,2202\{,\}220\-dimensional feature vectors per window\.

##### Joint PCA\(50\) embedding\.

To make the Vendi computation tractable and to apply a consistent similarity geometry across conditions, we fit a single PCA basis of dimension5050\. The basis is fit jointly on a stratified subsample of50,00050\{,\}000windows, drawn proportionally across every condition \(each prior at every noise level, plus the training\-data baseline\) so that no condition dominates the principal directions\. All windows are then projected into this shared5050\-dimensional embedding before the Vendi score is computed\.

##### Computing the Vendi score\.

LetX∈ℝn×50X\\in\\mathbb\{R\}^\{n\\times 50\}be the PCA\-embedded windows of a single rollout sample, and letXnX\_\{n\}denoteXXwith rowsℓ2\\ell\_\{2\}\-normalized\. We use the cosine similarity kernel and exploit the dual identityeig⁡\(Xn​Xn⊤/n\)=eig⁡\(Xn⊤​Xn/n\)\\mathrm\{eig\}\(X\_\{n\}X\_\{n\}^\{\\top\}/n\)=\\mathrm\{eig\}\(X\_\{n\}^\{\\top\}X\_\{n\}/n\)\(up to zero eigenvalues\) to compute Vendi via the small50×5050\\times 50Gram matrix,

M=1n​Xn⊤​Xn,\{λi\}i=150=eigvalsh⁡\(M\),M=\\frac\{1\}\{n\}\\,X\_\{n\}^\{\\top\}X\_\{n\},\\qquad\\\{\\lambda\_\{i\}\\\}\_\{i=1\}^\{50\}=\\mathrm\{eigvalsh\}\(M\),which is exact for the cosine Vendi score and avoids forming then×nn\\times nkernel\. The Vendi score is then the exponential of the Shannon entropy of the \(positive\) eigenvalues,

VS\(X\)=exp\(−∑i:λi\>0λilogλi\),\\mathrm\{VS\}\(X\)=\\exp\\\!\\left\(\-\\sum\_\{i:\\,\\lambda\_\{i\}\>0\}\\lambda\_\{i\}\\,\\log\\lambda\_\{i\}\\right\),with the convention0​log⁡0=00\\log 0=0\.

##### Bootstrap over clips\.

The independent unit of replication is the imitation*clip*\(windows within a clip share a starting condition and are not independent\), so we bootstrap at the clip level\. There are842842clips per condition \(motion\-prior, behavior\-prior, training\-data\)\. For each condition we bootstrap by resampling the clip indices with replacement and recompute the Vendi score\. We performB=500B=500bootstrap iterations\. The same clip indices are reused across all conditions in a given iteration, so cross\-policy comparisons are exactly paired\. We report bootstrap means with 95% confidence intervals\.

### F\.3Kernel Inception Distance

##### What KID measures\.

The Kernel Inception Distance \(KID\) quantifies the dissimilarity between two distributions in a learned feature space as the squared Maximum Mean Discrepancy \(MMD2\\mathrm\{MMD\}^\{2\}\) under a polynomial kernel\[[37](https://arxiv.org/html/2610.27279#bib.bib37)\]\. In contrast to a metric like Fréchet Inception Distance \(FID\)\[[49](https://arxiv.org/html/2610.27279#bib.bib49)\], it is an unbiased estimator and remains well\-defined for moderately sized samples\. KID is zero when the two distributions match\.

##### Training the feature extractor\.

There is no pre\-trained “Inception” network for rodent kinematics, so we train a small variational autoencoder \(VAE\) on the real \(training\-data\) joint\-angle trajectories and use its encoder meanμϕ​\(𝐱\)∈ℝd\\mu\_\{\\phi\}\(\\mathbf\{x\}\)\\in\\mathbb\{R\}^\{d\}as the feature extractor\. The VAE is optimized by the standardβ\\beta\-VAE objective

ℒVAE\(ϕ,θ\)=𝔼𝐱\[∥𝐱^θ\(𝐱\)−𝐱∥22\+βDKL\(qϕ\(𝐳∣𝐱\)∥𝒩\(𝟎,𝐈\)\)\],\\mathcal\{L\}\_\{\\mathrm\{VAE\}\}\(\\phi,\\theta\)=\\mathbb\{E\}\_\{\\mathbf\{x\}\}\\\!\\left\[\\\|\\hat\{\\mathbf\{x\}\}\_\{\\theta\}\(\\mathbf\{x\}\)\-\\mathbf\{x\}\\\|\_\{2\}^\{2\}\+\\beta\\,D\_\{\\mathrm\{KL\}\}\\\!\\left\(q\_\{\\phi\}\(\\mathbf\{z\}\\mid\\mathbf\{x\}\)\\,\\\|\\,\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{I\}\)\\right\)\\right\],with hyperparameters listed in Table[8](https://arxiv.org/html/2610.27279#A6.T8)\. After training, the decoder is discarded and onlyμϕ\\mu\_\{\\phi\}is retained; we repeat training under several random seeds and report the mean and standard deviation of KID across them\.

##### Trajectory inputs and preprocessing\.

Each training sample is a single 5 s trajectory of the rat’s generalized coordinates𝐪∈ℝ74\\mathbf\{q\}\\in\\mathbb\{R\}^\{74\}— planar root position\(x,y\)\(x,y\), root heightzz, root quaternion \(44dims\), and6767joint angles — drawn from the training\-data imitation rollouts\. We use imitation rollouts as the ground truth as opposed to the original mocap because the alignment process from keypoints to joint angle trajectories \(through STAC\-MJX\[[6](https://arxiv.org/html/2610.27279#bib.bib6)\]\) produces jagged transition frame\-to\-frame\. Imitation of this reference allows the simulated physics to smooth out the trajectories and results in a fairer comparison to the generated motion\. Before encoding, three operations are applied so that the comparison between training and generated distributions is invariant to nuisance factors and on a consistent scale: \(i\) the root quaternion is replaced by a yaw\-removed66D rotation representation\[[50](https://arxiv.org/html/2610.27279#bib.bib50)\]; \(ii\) the planar root position\(x,y\)\(x,y\)is discarded; \(iii\) joint dimensions arezz\-score normalized using statistics computed on the real data\. Each per\-step vector therefore consists of root height, root rotation, and6767normalized joint angles for7474features per step; the full trajectory is flattened to a single vector before being fed to the encoder\. The same preprocessing is applied to every generated rollout\.

##### Computing KID\.

LetX=\{μir\}i=1mX=\\\{\\mu\_\{i\}^\{r\}\\\}\_\{i=1\}^\{m\}andY=\{μjf\}j=1mY=\\\{\\mu\_\{j\}^\{f\}\\\}\_\{j=1\}^\{m\}be the encoder features of the real and generated samples\. We use the cubic polynomial kernel of[Bińkowski et al\. \[37\]](https://arxiv.org/html/2610.27279#bib.bib37),

k⁡\(𝐮,𝐯\)=\(1d​𝐮⊤​𝐯\+1\)3,k\(\\mathbf\{u\},\\mathbf\{v\}\)=\\left\(\\frac\{1\}\{d\}\\,\\mathbf\{u\}^\{\\top\}\\mathbf\{v\}\+1\\right\)^\{\\\!3\},and defineMMD^u2\\widehat\{\\mathrm\{MMD\}\}\_\{u\}^\{2\}as the unbiased squared\-MMD estimator of[Gretton et al\. \[51\]](https://arxiv.org/html/2610.27279#bib.bib51),

MMD^u2​\(X,Y\)=1m⁡\(m−1\)​∑i≠jk⁡\(μir,μjr\)\+1m⁡\(m−1\)​∑i≠jk⁡\(μif,μjf\)−2m2​∑i,jk⁡\(μir,μjf\)\.\\widehat\{\\mathrm\{MMD\}\}\_\{u\}^\{2\}\(X,Y\)=\\frac\{1\}\{m\(m\-1\)\}\\sum\_\{i\\neq j\}k\(\\mu\_\{i\}^\{r\},\\mu\_\{j\}^\{r\}\)\+\\frac\{1\}\{m\(m\-1\)\}\\sum\_\{i\\neq j\}k\(\\mu\_\{i\}^\{f\},\\mu\_\{j\}^\{f\}\)\-\\frac\{2\}\{m^\{2\}\}\\sum\_\{i,j\}k\(\\mu\_\{i\}^\{r\},\\mu\_\{j\}^\{f\}\)\.To stabilize the per\-seed estimate, we averageMMD^u2\\widehat\{\\mathrm\{MMD\}\}\_\{u\}^\{2\}overSSrandom subsets of sizemmdrawn independently fromXXandYY:

KID⁡\(X,Y\)=1S​∑s=1SMMD^u2​\(X\(s\),Y\(s\)\)\.\\mathrm\{KID\}\(X,Y\)=\\frac\{1\}\{S\}\\sum\_\{s=1\}^\{S\}\\widehat\{\\mathrm\{MMD\}\}\_\{u\}^\{2\}\\\!\\left\(X^\{\(s\)\},Y^\{\(s\)\}\\right\)\.
Table 8:Hyperparameters for the VAE feature extractor \(“inception network”\) used to compute KID\.GroupHyperparameterValueArchitectureEncoder hidden sizes\[512,256\]\[512,256\]Decoder hidden sizes\[256,512\]\[256,512\]Latent dimensionalitydd6464ActivationSiLULayer normalizationyesRegularizationDropout rate0\.10\.1Weight decay \(AdamW\)1×10−51\\times 10^\{\-5\}β\\beta\-VAETargetβ\\beta1\.01\.0β\\betawarmup \(linear\)2,5002\{,\}500epochsTrainingOptimizerAdamWLearning rate1×10−31\\times 10^\{\-3\}Gradient clip norm1\.01\.0Batch size128128Total epochs5,0005\{,\}000Random seeds\{0,1,2,3\}\\\{0,1,2,3\\\}KID computationPolynomial kernel degree33Number of subsetsSS100100Subset sizemmmin⁡\(\|X\|,\|Y\|,1,000\)\\min\(\|X\|,\|Y\|,1\{,\}000\)

## Appendix GGait analysis

We compare the gaits produced by SCAMPER, MIMIC\-MJX, and the from\-scratch baseline against empirical rat gait data from[Danner et al\. \[39\]](https://arxiv.org/html/2610.27279#bib.bib39), at three target speeds spanning walk, trot, and gallop\. The same pipeline is applied to every model rollout so that the only difference between conditions is the controller\.

##### Rollouts\.

For each \(model, target speed\) condition we run a fixed\-length deterministic rollout of the locomotion task, with the velocity setpoint chosen to elicit the corresponding gait\. The simulator state, the foot\-body world positions, and the four limb touch\-sensor traces \(palm and sole on each side\) are saved for offline analysis\.

##### Contact barcode\.

For each rollout we convert the four raw touch\-sensor traces into a binary stance/swing barcode of shape\[4,T\]\[4,T\], with rows ordered\[RH,RF,LF,LH\]\[\\text\{RH\},\\text\{RF\},\\text\{LF\},\\text\{LH\}\]\. The pipeline is: \(i\) per\-channel median filtering to suppress sensor noise; \(ii\) an optional spike\-removal step that zeros out brief above\-threshold transients on a sensor channel that is otherwise quiescent \(so that fleeting contacts never get registered as stance bouts\); \(iii\) per\-channel thresholding to a binary stance signal, with the threshold tuned per limb to account for differing typical contact forces; and \(iv\) binary morphological closing on each channel to fill brief gaps in otherwise\-continuous stance bouts\. The four cleaned binary traces are then stacked into the barcode used for all subsequent steps\.

##### Gait cycle detection\.

Gait cycles are extracted from the barcode using a chosen reference limb\. We find every stance onset \(a→10\\\!\\to\\\!1transition on the reference channel\), discard onsets that are closer together than a minimum cycle length \(rejects double\-detection of a single foot\-strike\), and define each gait cycle as the interval between two consecutive onsets that is at least the minimum cycle length\. Detection is restricted to a per\-condition analysis window so that the rat is at steady\-state speed during the cycles being analyzed\.

##### Per\-cycle metrics\.

For every detected cycle we compute the standard set of behavioral\-gait metrics used in rodent locomotion studies\[[40](https://arxiv.org/html/2610.27279#bib.bib40),[41](https://arxiv.org/html/2610.27279#bib.bib41)\]:

- •Stride length\.Euclidean distance between the reference foot’s world position at the start and end of the cycle\.
- •Mean velocity\.Stride length divided by cycle duration\.
- •Cadence\.Stride frequency converted to steps per minute \(counting both feet of a contralateral pair\)\.
- •Stance percentage / duty factor\.Fraction of the cycle during which the reference limb is in stance\.

We then aggregate these metrics across all cycles in the analysis window and report mean and standard deviation per \(model, gait\) condition\.

##### Swing–stance gait diagrams\.

The diagrams in Figure[4](https://arxiv.org/html/2610.27279#S4.F4)c are computed by isolating, on each of the four limb channels, every complete stance bout within the analysis window \(partial bouts that overlap the window edges are dropped\), then computing per\-limb mean stance duration and standard deviation, mean stride period \(median inter\-onset interval averaged over limbs\), and the phase offset of each limb relative to a reference hind limb\. Phase offsets are taken as the mean delay between the first stance onset of each limb after a reference\-limb onset, modulo the stride period\. The resulting per\-limb stance bars are tiled across the cycle for visualization\.

##### Empirical reference\.

The empirical rat data\[[39](https://arxiv.org/html/2610.27279#bib.bib39)\]are filtered to intact\-spinal\-cord animals for each of the three gait categories, and we extract per\-stride duty factors for the four limbs, the inter\-limb phase relationships \(left–right hind, homolateral, and diagonal\), and the stride duration\. These are converted into the same set of per\-limb mean stance, phase offset, mean stride period, and duty\-factor summaries used for the model rollouts, providing a directly comparable reference distribution\.

##### Quantitative summary\.

For the comparison table, every metric is summarized per \(gait, model\) asμ±σ\\mu\\pm\\sigmaover cycles\. Each model is then scored against the empirical reference as the difference in means normalized by the empirical standard deviation,\|Δ​μ\|/σemp\|\\Delta\\mu\|/\\sigma\_\{\\text\{emp\}\}, which expresses the discrepancy in units of the natural across\-stride variability of real rats\. The metrics included are stride duration, step frequency, per\-limb stance and swing duration, per\-limb duty factor, temporal symmetry \(left/right hind duty\-factor asymmetry\), and per\-limb phase offset — giving a single table that aggregates the full per\-limb timing pattern alongside the cycle\-level summaries\.

## Appendix HSupplementary analyses and results

### H\.1Empirical tests of Propositions 1 and 2

To evaluate the encoder and the two priors on the states they actually encounter, we rolled out the reference dataset in closed loop through the frozen Stage\-2 encoder–decoder and collected ~200,000 proprioceptive states\. From these we drew 20,000 anchor states\. For each anchor we identified the 32 most similar proprioceptive states in a 64\-dimensional principal\-component space, excluding the anchor’s own clip to avoid autocorrelation, and took the goal that followed each of them as a candidate goal\. Each anchor’s proprioception was then held fixed while its goal was swapped for each of theseK=32K=32candidates\.

Both propositions rest on the identity𝔼\[DKL\(qϕ∥p\)\]=Iϕ\(Z;G∣X\)\+𝔼\[DKL\(mϕ∥p\)\]\\mathbb\{E\}\\left\[D\_\{\\mathrm\{KL\}\}\(q\_\{\\phi\}\\,\\\|\\,p\)\\right\]=I\_\{\\phi\}\(Z;G\\mid X\)\+\\mathbb\{E\}\\left\[D\_\{\\mathrm\{KL\}\}\(m\_\{\\phi\}\\,\\\|\\,p\)\\right\], wheremϕm\_\{\\phi\}is the encoder marginal\. The first term depends only on the encoder, so it is identical for both priors and cancels in their difference\. The comparison between priors therefore reduces to closed\-form Gaussian KL divergences\. The behavior priorpφp\_\{\\varphi\}is 8\.46 nats closer to the encoder marginal than the motion priorpψp\_\{\\psi\}averaged across all individual states \(Fig\.[7](https://arxiv.org/html/2610.27279#A8.F7)a\)\.

To measure how much of the encoder’s behavior at a fixed body state is driven by the goal, we decompose the encoder’s latent variance at each anchor with the law of total variance\. The between\-goal term is the variance of the encoder mean across the 32 candidate goals,Varg​\[μϕ\]\\mathrm\{Var\}\_\{g\}\[\\mu\_\{\\phi\}\], and the within\-goal term is the expected encoder variance,𝔼g​\[σϕ2\]\\mathbb\{E\}\_\{g\}\[\\sigma\_\{\\phi\}^\{2\}\]\. With proprioception held fixed, 49\.7% of the encoder’s latent variance is between\-goal \(Fig\.[7](https://arxiv.org/html/2610.27279#A8.F7)b\)\. The regularization in Proposition 1 therefore limits the conditional mutual informationIϕ​\(Z;G∣X\)I\_\{\\phi\}\(Z;G\\mid X\)but does not drive it to zero\. Driving it to zero would be undesirable, because the behavior prior distilled from the encoder could then no longer produce diverse motion\.

Because all 32 candidates at an anchor share the same proprioceptive state, any above\-chance decoding of the goal reflects information carried by the latent alone\. We draw one candidate uniformly at random, sample a latent from the encoder’s distribution for that candidate, score all 32 candidates by the density their encoder Gaussians assign to the sample, and predict the highest\-scoring one\. Since the candidate prior is uniform and the densities are the encoder’s own, this rule is Bayes\-optimal and needs no trained probe\. It identifies the correct goal 69\.4% of the time, against 3\.1% chance \(a 22\-fold improvement\), and ranks it in the top 5 for 94\.6% of anchors, against 15\.6% chance \(Fig\.[7](https://arxiv.org/html/2610.27279#A8.F7)c\)\.

Finally, we tested Proposition 1 directly by repeating the fixed\-proprioception analysis on all 50 Stage\-2 checkpoints, each with its own observation normalizer\. We measure goal sensitivity as the mutual information between the latent and which of theKKcandidate goals was used, which is bounded above bylog⁡K\\log K\. The Stage\-2 KL coefficientλ\\lambdaramps linearly down from10−310^\{\-3\}to10−410^\{\-4\}over the first 25% of training and is then held at its floor\. Goal sensitivity tracks this schedule\. It falls from 1\.70 to 1\.33 nats whileλ\\lambdais near its maximum, which is the bottleneck Proposition 1 describes\. It then rises monotonically to 2\.52 nats asλ\\lambdadecays and stops changing whenλ\\lambdareaches its floor, varying by less than 0\.03 nats over the remaining 75% of training \(Fig\.[7](https://arxiv.org/html/2610.27279#A8.F7)d\)\. Nothing else in the training schedule changes at that point, so the strength of the bottleneck, rather than generic training dynamics, sets how much goal information the latent carries\. Proposition 1 should therefore be read as describing a pressure that acts during training rather than a property of the final model: the schedule deliberately relaxes the bottleneck to improve reconstruction, and the converged encoder is goal\-rich\.

Figure 7:\(a\)KL divergence to the encoder marginal for the motion and behavior priors\.\(b\)Between\-goal and within\-goal shares of the encoder’s latent variance at fixed proprioception\.\(c\)Goal decoding accuracy from the latent, compared with chance\.\(d\)Goal sensitivity across Stage\-2 training, with the KL coefficientλ\\lambdaon the right axis\. Error bars are 95% clip\-level bootstrap CIs\.
### H\.2Diversity and realism of prior sampling

In Fig\.[2](https://arxiv.org/html/2610.27279#S4.F2)we demonstrate that when sampling random motion trajectories, we obtain greater behavioral diversity from the behavior prior than from the motion prior quantified by the resulting Vendi scores\. However, the observed difference in diversity could reflect discrepancies in effective perturbation magnitude of the latent rather than qualitatively different distributions\. To address this, we compare the behavioral diversity by examining the impact of noise as a function of both the scales and the jitter of the sampled latents\. To disambiguate whether the scale of the latents is in fact responsible for the difference in diversity, we measure the Vendi score against the mean per\-dimension standard deviation of the sampled latents for both priors across all rollouts at various sampling noise levels \(Fig\.[8](https://arxiv.org/html/2610.27279#A8.F8)a\)\. While we do find that the scale of the latents is different for the two priors, when accounting for this along the domain of the measured latent variance, the behavior prior is still more diverse than the motion prior\. Secondly, to disambiguate whether the behavior prior’s increased diversity is a result of jitter \(i\.e\., how quickly the latents change from timestep to timestep\), we measure the Vendi score versus the one\-step latent displacement \(Fig\.[8](https://arxiv.org/html/2610.27279#A8.F8)b\)\. While we again find that both priors tend towards varying rates of per\-step latent displacement for the same sampled noise level, when accounting for this difference, the behavior prior remains more behaviorally diverse than the motion prior over the domain of the latent jitter measured\.

Figure 8:Diversity at matched latent scale\. Vendi score of random generations vs\. the realized per\-dimension latent s\.d\. \(a\) and mean one\-step latent displacement \(b\) at various levels of sampling noise \(σ\\sigma\)\. Stars: deterministic \(σ=0\\sigma=0\)\. 95% bootstrap CIs shown as shaded regions\.![Refer to caption](https://arxiv.org/html/2610.27279v1/supp3.png)Figure 9:Spatial occupancy of all tested sampling noise levels plotted as 2D histograms of body location \(log\-scaled frame counts\) for the motion prior and behavior prior\.![Refer to caption](https://arxiv.org/html/2610.27279v1/supp4.png)Figure 10:Empirical cumulative distribution function of root height at all tested sampling noise levels compared against the training data reference \(dashed gray\)\.Table 9:Wasserstein distance between empirical root\-height distributions of prior rollouts and the training data, across sampling noise levelsσ\\sigma\. Corresponds to results from Figure[10](https://arxiv.org/html/2610.27279#A8.F10)\.
### H\.3Foraging task

#### H\.3\.1Arena foraging supplemental results

![Refer to caption](https://arxiv.org/html/2610.27279v1/supp5.png)Figure 11:Rendering of the arena foraging task\.![Refer to caption](https://arxiv.org/html/2610.27279v1/supp6.png)Figure 12:SCAMPER’s training progression on the arena foraging task\. Top\-down view of trajectories the rodent takes along with the evolution of the gate logit \(βt\\beta\_\{t\}\) and its resulting binarization \(btb\_\{t\}\)\. Here when a treat is collected it is still displayed in the arena for visualization purposes, but in practice it disappear upon successful collection\.bt=0b\_\{t\}=0results in stochastic sampling of the behavior prior\.bt=1b\_\{t\}=1results in residual control of the motion prior\. During early training the rodent relies heavily on the behavior prior to explore its environment\. During mid training, the rodent begins to gate rapidly between both priors as it begins to exploit the reward signals from the collection of treats\. During late training the rodent mostly uses residual control of the motion prior to effectively exploit the task structure\.
#### H\.3\.2Alternatives to the adaptive binary gate

The adaptive binary gate is one way to decide when the behavior prior or the motion prior controls the latent\. To test whether the specific mechanism matters, we replaced it in the foraging task with two alternatives that need no learned gate: a fixed exploration schedule and a fixed linear interpolation between the two priors\. In both variants the policy outputs only the residual, and the motion prior, behavior prior and decoder stay frozen, as with the adaptive gate\.

##### Fixed exploration schedule\.

The latent is constructed exactly as with the adaptive gate,𝐳t=gt​\(𝝁tMP\+𝜹t\)\+\(1−gt\)​𝐳tBP\\mathbf\{z\}\_\{t\}=g\_\{t\}\\,\(\\boldsymbol\{\\mu\}^\{\\mathrm\{MP\}\}\_\{t\}\+\\boldsymbol\{\\delta\}\_\{t\}\)\+\(1\-g\_\{t\}\)\\,\\mathbf\{z\}^\{\\mathrm\{BP\}\}\_\{t\}, where𝝁tMP\\boldsymbol\{\\mu\}^\{\\mathrm\{MP\}\}\_\{t\}is the motion prior mean,𝜹t\\boldsymbol\{\\delta\}\_\{t\}is the policy’s residual and𝐳tBP\\mathbf\{z\}^\{\\mathrm\{BP\}\}\_\{t\}is a behavior prior sample\. However, the binary gategtg\_\{t\}is no longer emitted by the policy\. Instead, it switches stochastically, and the probabilityp⁡\(gt=1\)p\(g\_\{t\}=1\)of motion\-prior control follows a fixed schedule over training\. It is held at00for an initial warmup fraction of training, ramped to11over a subsequent ramp fraction, and held at11thereafter\. We vary when this switch happens\.

##### Linear interpolation\.

Instead of selecting one prior at each step, the two priors are blended with a constant weightρ∈\[0,1\]\\rho\\in\[0,1\]and the residual is added on top:𝐳t=ρ​𝝁tMP\+\(1−ρ\)​𝐳tBP\+𝐫t\\mathbf\{z\}\_\{t\}=\\rho\\,\\boldsymbol\{\\mu\}^\{\\mathrm\{MP\}\}\_\{t\}\+\(1\-\\rho\)\\,\\mathbf\{z\}^\{\\mathrm\{BP\}\}\_\{t\}\+\\mathbf\{r\}\_\{t\}\. Hereρ\\rhois a fixed hyperparameter rather than a learned quantity\. The endpointsρ=1\\rho=1andρ=0\\rho=0reduce to residual control of the motion prior alone and of the behavior prior alone, respectively\.

##### Results\.

Table[10](https://arxiv.org/html/2610.27279#A8.T10)reports the mean number of treats collected in the hardest foraging condition \(minimum treat radius of1\.51\.5m\)\. Both alternatives can succeed, but only within a narrow range of settings\. The fixed schedule trains successfully when control passes from the behavior prior to the motion prior early in training \(warmup0\.050\.05, ramp0\.250\.25, so that the motion prior has full control by30%30\\%of training\)\. It fails when the switch happens in the middle or later phases of training, and training also fails if the switch never happens\. Likewise, interpolation succeeds only when the blend is weighted towards the behavior prior \(ρ=0\.25\\rho=0\.25\) and fails forρ=0\.5\\rho=0\.5andρ=0\.75\\rho=0\.75\.

These results show that what matters is the right mix of the behavior and motion priors over training\. The adaptive binary gate is one means of achieving that mix, not the only one\. Its advantage over the alternatives is that the user does not need to hand\-tune a schedule or blend weight: the gate only needs to be biased towards the behavior prior at the start of training, and the transition to the motion prior then emerges from learning \(Figure[12](https://arxiv.org/html/2610.27279#A8.F12)\)\.

Table 10:Alternatives to the adaptive binary gate in the foraging task\. Values are the mean number of treats collected in the hardest condition \(minimum treat radius1\.51\.5m\)\. Schedule fractions are of total training steps\.α\\alphais the weight on the motion prior\.

### H\.4Locomotion task

Here we present the full gait analysis results for the empirical rat gait data\[[39](https://arxiv.org/html/2610.27279#bib.bib39)\], the main SCAMPER model which residually controls the motion prior \(MP\), a control which residually controls the behavior prior \(BP\), MIMIC\-MJX, and the from\-scratch trained model\. Notably, when performing residual control using the behavior prior, galloping is never discovered and the lower speed gaits also lack resemblance to the empirical data both qualitatively \(Fig\.[13](https://arxiv.org/html/2610.27279#A8.F13)\) and quantitatively \(Table[11](https://arxiv.org/html/2610.27279#A8.T11)\)\.

Figure 13:Stance\-phase plots for each limb \(RF, LF, RH, LH\) across walk, trot, and gallop gaits, comparing empirical data \(purple;\[[39](https://arxiv.org/html/2610.27279#bib.bib39)\]\) to the base SCAMPER model \(MP,blue\), the behavior\-prior\-controlled SCAMPER model \(BP,dark blue\), MIMIC\-MJX \(red\), and the from\-scratch model \(gray\); bars show stance duration, error bars show variability across strides\.Table 11:Full gait metric comparison to empirical rat data\. Empirical, SCAMPER \(MP\), SCAMPER \(BP\), MIMIC\-MJX, and Scratch columns show mean±\\pmstd\.\|Δ​μ\|/σemp\|\\Delta\\mu\|/\\sigma\_\{\\text\{emp\}\}is the absolute difference in means normalized by empirical standard deviation\.EmpiricalSCAMPER \(MP\)SCAMPER \(BP\)MIMIC\-MJXScratch\|Δ​μ\|/σemp\|\\Delta\\mu\|/\\sigma\_\{\\text\{emp\}\}\(↓\\downarrow\)GaitMetricSCAMPER \(MP\)SCAMPER \(BP\)MIMIC\-MJXScratchWalkStride Duration \(ms\)308±\\pm91332±\\pm16313±\\pm34374±\\pm23255±\\pm2930\.270\.060\.730\.58Step Frequency \(Hz\)3\.46±\\pm0\.753\.02±\\pm0\.153\.22±\\pm0\.332\.69±\\pm0\.264\.61±\\pm2\.620\.580\.311\.021\.53Stance LH \(ms\)182±\\pm82191±\\pm21178±\\pm26221±\\pm36223±\\pm2950\.110\.050\.480\.50Stance RH \(ms\)175±\\pm66183±\\pm9208±\\pm59247±\\pm3180±\\pm00\.120\.511\.091\.45Stance LF \(ms\)171±\\pm71172±\\pm32137±\\pm11153±\\pm1520±\\pm100\.020\.480\.252\.12Stance RF \(ms\)174±\\pm69183±\\pm6217±\\pm13187±\\pm160±\\pm00\.130\.620\.192\.53Swing LH \(ms\)125±\\pm20144±\\pm5128±\\pm7152±\\pm10150±\\pm450\.960\.111\.331\.23Swing RH \(ms\)133±\\pm31148±\\pm5113±\\pm21123±\\pm8175±\\pm00\.510\.650\.291\.37Swing LF \(ms\)136±\\pm25152±\\pm7174±\\pm9219±\\pm22120±\\pm00\.631\.523\.330\.66Swing RF \(ms\)133±\\pm30149±\\pm394±\\pm5184±\\pm7255±\\pm00\.501\.301\.673\.99Duty Factor LH\.576±\\pm\.074\.576±\\pm\.064\.568±\\pm\.082\.593±\\pm\.095\.877±\\pm1\.1570\.000\.110\.234\.08Duty Factor RH\.562±\\pm\.055\.550±\\pm\.027\.665±\\pm\.188\.660±\\pm\.083\.314±\\pm\.0000\.211\.871\.774\.47Duty Factor LF\.544±\\pm\.059\.519±\\pm\.098\.438±\\pm\.036\.409±\\pm\.039\.079±\\pm\.0390\.441\.822\.317\.96Duty Factor RF\.558±\\pm\.055\.552±\\pm\.020\.691±\\pm\.041\.502±\\pm\.042\.000±\\pm\.0000\.112\.421\.0310\.14Temporal Symmetry\.011±\\pm\.068\.016±\\pm\.024−\-\.050±\\pm\.067−\-\.049±\\pm\.074\.000±\\pm\.0000\.080\.890\.880\.16Phase RH rel\. LH\.459±\\pm\.041\.513±\\pm\.113\.539±\\pm\.104\.368±\\pm\.094\.432±\\pm\.0001\.321\.962\.250\.67Phase LF rel\. LH\.333±\\pm\.047\.459±\\pm\.094\.428±\\pm\.062\.516±\\pm\.102\.334±\\pm\.2952\.692\.043\.930\.02Phase RF rel\. LH\.824±\\pm\.045\.906±\\pm\.165\.755±\\pm\.077\.835±\\pm\.206\.000±\\pm\.0001\.861\.530\.2618\.48Mean \(Coordination\)1\.491\.601\.834\.83Mean \(Gait Pattern\)0\.951\.681\.686\.55Mean \(All\)0\.591\.011\.283\.44TrotStride Duration \(ms\)230±\\pm31267±\\pm10130±\\pm4285±\\pm92318±\\pm3911\.193\.241\.762\.84Step Frequency \(Hz\)4\.42±\\pm0\.563\.75±\\pm0\.227\.71±\\pm0\.264\.74±\\pm2\.814\.41±\\pm2\.521\.185\.840\.570\.01Stance LH \(ms\)97±\\pm27135±\\pm1660±\\pm2181±\\pm4820±\\pm01\.461\.383\.182\.88Stance RH \(ms\)97±\\pm25134±\\pm670±\\pm387±\\pm501251±\\pm23211\.491\.100\.4046\.48Stance LF \(ms\)104±\\pm2393±\\pm538±\\pm8116±\\pm1810±\\pm00\.512\.930\.504\.16Stance RF \(ms\)104±\\pm24118±\\pm551±\\pm4155±\\pm2720±\\pm00\.602\.252\.153\.56Swing LH \(ms\)134±\\pm14129±\\pm370±\\pm3135±\\pm26110±\\pm100\.334\.640\.061\.72Swing RH \(ms\)133±\\pm15135±\\pm560±\\pm380±\\pm14190±\\pm430\.104\.983\.673\.88Swing LF \(ms\)126±\\pm16175±\\pm592±\\pm9211±\\pm22308±\\pm03\.132\.185\.4411\.69Swing RF \(ms\)126±\\pm16150±\\pm479±\\pm4173±\\pm7298±\\pm01\.472\.932\.8910\.62Duty Factor LH\.413±\\pm\.065\.506±\\pm\.061\.460±\\pm\.012\.636±\\pm\.169\.063±\\pm\.0001\.430\.723\.435\.42Duty Factor RH\.416±\\pm\.060\.502±\\pm\.024\.537±\\pm\.025\.306±\\pm\.1763\.931±\\pm7\.2911\.422\.021\.8558\.61Duty Factor LF\.450±\\pm\.053\.348±\\pm\.017\.292±\\pm\.061\.407±\\pm\.064\.031±\\pm\.0001\.953\.010\.837\.96Duty Factor RF\.448±\\pm\.055\.442±\\pm\.017\.391±\\pm\.030\.543±\\pm\.094\.063±\\pm\.0000\.111\.021\.716\.95Temporal Symmetry−\-\.004±\\pm\.058\.008±\\pm\.068−\-\.075±\\pm\.017\.172±\\pm\.233−\-\.434±\\pm\.1620\.211\.233\.067\.46Phase RH rel\. LH\.488±\\pm\.060\.543±\\pm\.092\.308±\\pm\.045\.474±\\pm\.210\.401±\\pm\.3470\.912\.980\.241\.45Phase LF rel\. LH\.473±\\pm\.046\.620±\\pm\.075\.406±\\pm\.044\.699±\\pm\.121\.000±\\pm\.0003\.211\.474\.9410\.35Phase RF rel\. LH\.975±\\pm\.070\.921±\\pm\.173\.900±\\pm\.142\.108±\\pm\.070\.126±\\pm\.0000\.771\.0712\.3612\.11Mean \(Coordination\)1\.271\.695\.157\.84Mean \(Gait Pattern\)1\.401\.763\.6214\.69Mean \(All\)1\.192\.502\.7211\.01GallopStride Duration \(ms\)192±\\pm19175±\\pm6140±\\pm5255±\\pm91172±\\pm1000\.852\.673\.321\.01Step Frequency \(Hz\)5\.27±\\pm0\.535\.71±\\pm0\.237\.15±\\pm0\.255\.22±\\pm2\.547\.95±\\pm3\.110\.833\.520\.095\.04Stance LH \(ms\)78±\\pm1256±\\pm675±\\pm988±\\pm5315±\\pm81\.830\.220\.815\.32Stance RH \(ms\)74±\\pm1366±\\pm653±\\pm575±\\pm5016±\\pm170\.621\.610\.024\.33Stance LF \(ms\)76±\\pm1131±\\pm432±\\pm5138±\\pm400±\\pm04\.003\.925\.496\.80Stance RF \(ms\)84±\\pm1647±\\pm655±\\pm5141±\\pm280±\\pm02\.301\.763\.575\.20Swing LH \(ms\)114±\\pm13119±\\pm365±\\pm8115±\\pm46172±\\pm880\.453\.760\.124\.52Swing RH \(ms\)117±\\pm10108±\\pm687±\\pm593±\\pm13140±\\pm1030\.862\.932\.402\.24Swing LF \(ms\)115±\\pm11144±\\pm5108±\\pm5189±\\pm30172±\\pm02\.550\.666\.535\.07Swing RF \(ms\)108±\\pm9128±\\pm685±\\pm5187±\\pm9172±\\pm02\.162\.458\.426\.82Duty Factor LH\.406±\\pm\.042\.321±\\pm\.033\.538±\\pm\.061\.343±\\pm\.207\.087±\\pm\.0442\.043\.141\.527\.63Duty Factor RH\.386±\\pm\.040\.377±\\pm\.037\.377±\\pm\.034\.292±\\pm\.197\.094±\\pm\.0980\.230\.242\.377\.36Duty Factor LF\.398±\\pm\.032\.179±\\pm\.025\.230±\\pm\.037\.540±\\pm\.157\.000±\\pm\.0006\.755\.174\.4112\.28Duty Factor RF\.433±\\pm\.048\.266±\\pm\.036\.394±\\pm\.036\.552±\\pm\.110\.000±\\pm\.0003\.470\.822\.468\.99Temporal Symmetry\.025±\\pm\.059−\-\.084±\\pm\.060\.173±\\pm\.062\.086±\\pm\.311−\-\.095±\\pm\.2541\.832\.491\.022\.03Phase RH rel\. LH\.789±\\pm\.073\.692±\\pm\.084\.697±\\pm\.102\.446±\\pm\.183\.540±\\pm\.2981\.331\.264\.733\.44Phase LF rel\. LH\.560±\\pm\.037\.486±\\pm\.034\.790±\\pm\.047\.495±\\pm\.211\.000±\\pm\.0001\.996\.261\.7615\.20Phase RF rel\. LH\.188±\\pm\.062\.202±\\pm\.044\.227±\\pm\.044\.168±\\pm\.037\.000±\\pm\.0000\.230\.640\.323\.05Mean \(Coordination\)1\.352\.661\.965\.93Mean \(Gait Pattern\)2\.292\.502\.518\.28Mean \(All\)1\.912\.422\.745\.91