Amortized Low-Rank Adaptation for Model-Based Reinforcement Learning

# Amortized Low-Rank Adaptation for Model-Based Reinforcement Learning
Source: [https://arxiv.org/html/2609.12278](https://arxiv.org/html/2609.12278)
Fernando Palafox & David Fridovich\-KeilAffiliation:University of Texas at AustinAffiliation:Austin, TX, USAEmail:[\{fernandopalafox,dfk\}@utexas\.edu](mailto:)

###### Abstract

World models let agents plan by predicting the consequences of their actions, but changes in the environment can make them inaccurate\. We study the problem of adapting a world model to an unknown test\-time environment, drawn from a known environment family, using only a few episodes of interaction\. Existing approaches trade off computational cost against expressivity, i\.e\., the range of models a method can produce\. For example, in\-context learning is computationally cheap but limited in expressivity, and gradient\-based adaptation is expressive but computationally expensive\. We presentCLAW\(Context\-conditionedLow\-rankAdaptation ofWorld models\), which addresses this tradeoff by using a hypernetwork to generate low\-rank \(LoRA\) adapters at test time\. During pretraining, we simulate adaptation to a variety of environments and jointly train the hypernetwork and base world model\. At test time, we freeze the base model and use a forward pass of the hypernetwork to generate adapters from a small batch of test\-time transitions\. We evaluateCLAWin locomotion and manipulation environment families that vary in dynamics, embodiment, and reward\. We show that, using only seconds of test\-time data,CLAWoutperforms gradient\-based adaptation and in\-context learning during online adaptation\. We also show thatCLAWavoids overfitting in data\-scarce regimes, that its advantage comes from the expressive adapters rather than context conditioning, and that pretraining the hypernetwork jointly with the base model outperforms training it post hoc\.

## 1Introduction

In reinforcement learning \(RL\), learned world models allow agents to plan by predicting the consequences of their actions\([Ha and Schmidhuber, 2018](https://arxiv.org/html/2609.12278#bib.bib11);[Hafner et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib13);[Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14)\)\. However, changes in the environment can make a world model inaccurate, and it is often desirable to adapt it without collecting large test\-time datasets\. In this paper, we study the problem of adapting a world model to an unknown test\-time environment drawn from a known environment family \(e\.g\., a set of locomotion environments with varying dynamics, embodiments, or rewards\)\.

We cast this problem as a meta\-reinforcement learning problem where we amortize adaptation during pretraining, enabling online adaptation with little data\. Existing approaches face a tradeoff: they are either computationally cheap but limited in expressivity \(e\.g\., in\-context learning\), or expressive but computationally expensive \(e\.g\., gradient\-based weight adaptation\)\. We presentCLAW\(Context\-conditionedLow\-rankAdaptation ofWorld models\), an adaptation method which addresses this tradeoff\.CLAWuses hypernetworks\([Ha et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib10)\), neural networks trained to produce weights for other neural networks, to produce low\-rank \(LoRA\) adapters\([Hu et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib16)\)for the world model using a small batch of test\-time transitions\. This enables adaptation with the computational efficiency of in\-context learning while retaining the expressivity of weight\-space adaptation\.

We validateCLAWwith experiments in a variety of locomotion and manipulation environment families, where we vary dynamics, embodiments and rewards\. We show thatCLAWoutperforms gradient\-based and in\-context learning baselines during online adaptation, that it avoids overfitting in data\-scarce regimes, that its advantage comes from the expressivity of the generated adapters \(rather than context conditioning\), and that pretraining the hypernetwork jointly with the base model outperforms training it post hoc\.

## 2Related Work

### 2\.1Adaptation in Model\-Based RL

There are three main approaches to adapt a world model to test\-time environments\. The first approach changes the model’s weights directly by, e\.g\., gradient\-based re\-training\([Levy et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib62);[Levy et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib26);[Xu et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib40);[Pfrommer et al\., 2020](https://arxiv.org/html/2609.12278#bib.bib33);[Lanier et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib51);[Kim et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib58);[Wang et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib59)\); the same approach, applied to policies rather than world models, also appears in model\-free RL\([Ball et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib53);[Kostrikov et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib52);[Yin et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib50);[Amin et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib60)\)\. This approach is highly expressive \(i\.e\., can produce a large range of models\) at the cost of increased test\-time compute and high sample complexity\.

The second approach infers a learned embedding of the test\-time environment \(e\.g\., a latent task variable, a belief, or context embedding\) and conditions a fixed model on it\([Rakelly et al\., 2019](https://arxiv.org/html/2609.12278#bib.bib36);[Zintgraf et al\., 2019](https://arxiv.org/html/2609.12278#bib.bib45);[Zintgraf et al\., 2018](https://arxiv.org/html/2609.12278#bib.bib44);[Duan et al\., 2016](https://arxiv.org/html/2609.12278#bib.bib46);[Mishra et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib29);[James et al\., 2018](https://arxiv.org/html/2609.12278#bib.bib18);[Kumar et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib23);[Yu et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib42);[Zhang et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib43);[Chen et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib6);[Reed et al\., 2022](https://arxiv.org/html/2609.12278#bib.bib37);[Jiang et al\., 2022](https://arxiv.org/html/2609.12278#bib.bib20);[Laskin et al\., 2022](https://arxiv.org/html/2609.12278#bib.bib24);[Liu et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib27)\)\. This approach is computationally inexpensive \(embedding inference is often a single forward pass of a neural network\) but limited in expressivity since the weights are fixed at test time\.

A third approach skips test\-time adaptation entirely\. Instead, it trains one model to be robust across an entire family of environments\([Tobin et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib47);[Peng et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib31);[Andrychowicz et al\., 2020](https://arxiv.org/html/2609.12278#bib.bib2);[Lee et al\., 2020](https://arxiv.org/html/2609.12278#bib.bib25)\)\. This approach does not require test\-time adaptation mechanisms, but sacrifices accuracy on the test\-time environment for robustness across the whole family\.

Our approach,CLAW, is a combination of the first and second approaches\. We use a test\-time context embedding to generate low\-rank adapters\([Hu et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib16)\)that modify the model weights\.CLAWtakes the best of both approaches: model adaptation is nearly instantaneous \(a single forward pass of the hypernetwork\), and has the expressivity of full\-weight finetuning, depending on the adapter rank\.

### 2\.2Selecting What to Adapt

During weight\-based model adaptation, it is not always clear what weights within the model should be adapted\. Some approaches adapt all the weights, e\.g\., TD\-MPC2 adapts the entire model during online learning\([Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14)\)\. This has the benefit of maximum adaptation expressivity at the cost of increased test\-time computation and potential model collapse\([Smith et al\., 2022a](https://arxiv.org/html/2609.12278#bib.bib48);[Smith et al\., 2022b](https://arxiv.org/html/2609.12278#bib.bib49);[Yin et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib50)\)\. Other approaches adapt specific world model modules, e\.g\., the dynamics module\([Levy et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib26)\), or specific layers, e\.g\., the last layers\([Levy et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib62);[Harrison et al\., 2018](https://arxiv.org/html/2609.12278#bib.bib15);[Davydov et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib8)\)\. These approaches are computationally cheaper and avoid model collapse at the expense of manual module/layer selection and reduced adaptation expressivity\. A third approach does not modify weights and instead modulates the model’s intermediate activations, e\.g\., FiLM’s feature\-wise affine transforms\([Perez et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib32)\)\. This also avoids manual selection, but limits adaptation to modulation of existing activations rather than a full weight\-space update, which is more expressive\([Jayakumar et al\., 2020](https://arxiv.org/html/2609.12278#bib.bib19)\)\.

Our approach is different because it inserts LoRA adapters\([Hu et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib16)\)into all layers of the world model\. Therefore, we maintain most of the adaptation expressivity of full\-model adaptation at a fraction of test\-time compute cost\. Moreover, the pretraining procedure \([Algorithm1](https://arxiv.org/html/2609.12278#alg1)\) implicitly teaches the hypernetwork how strongly to adapt each layer, obviating the need for manual module/layer selection\.

### 2\.3Hypernetworks in RL

Hypernetworks have been used in meta\-RL before\([Finn et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib9);[Nagabandi et al\., 2018](https://arxiv.org/html/2609.12278#bib.bib63);[Raghu et al\., 2019](https://arxiv.org/html/2609.12278#bib.bib35)\)to replace test\-time gradient descent with a forward pass of a neural network\([Huang et al\., 2020](https://arxiv.org/html/2609.12278#bib.bib17);[Xian et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib39);[Przewięźlikowski et al\., 2022](https://arxiv.org/html/2609.12278#bib.bib34);[Beck et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib57);[Rezaei\-Shoshtari et al\., 2022](https://arxiv.org/html/2609.12278#bib.bib38);[Beukman et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib3)\)\. These methods often produce full weight tensors \(instead of LoRA adapters\)\.

In the domain of large language models \(LLMs\), existing work generates LoRA adapters from text prompts\([Ye and Ren, 2021](https://arxiv.org/html/2609.12278#bib.bib41);[Mahabadi et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib22);[Ortiz\-Barajas et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib30);[Charakorn et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib5);[Chen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib7);[Jukić et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib21);[Liu et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib28)\)\. A recent paper\([Bianchi et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib4)\)proposes generating LoRA adapters for a robot policy from a language instruction and a demo video\. These are all forms of amortized optimization\([Amos, 2023](https://arxiv.org/html/2609.12278#bib.bib61)\)in which a hypernetwork is trained to amortize test\-time low\-rank adaptation\.

Existing hypernetwork approaches either generate full model weights \(which is expensive and/or intractable for large world models\), or generate adapters from explicit task descriptions for a fixed pretrained model\. Neither approach addresses efficient world model adaptation from test\-time transitions\.CLAWfills this gap by jointly pretraining the world model and hypernetwork to infer adapters directly from test\-time interactions, enabling repeated, low\-cost adaptation during deployment\.

## 3Preliminaries

Each environmenteeis a Markov Decision Process \(MDP\), which we assume is drawn at test time from a uniform distributionp⁡\(ℰ\)p\(\\mathcal\{E\}\)\. For example,p⁡\(ℰ\)p\(\\mathcal\{E\}\)could assign probabilities to bipedal locomotion environments that differ in actuator strength\. At each steptt, the agent receives statests\_\{t\}, selects an action, and receives rewardrtr\_\{t\}together with the next statest\+1s\_\{t\+1\}\.

We define two functions that produce the agent’s actions: policies and planners\. Policies produce actions directly from states, i\.e\.,at=π⁡\(st\)a\_\{t\}=\\pi\(s\_\{t\}\), and are often instantiated as neural networks\. In contrast, planners select an action by evaluating candidate actions using a learned model of the MDP\. That is, given world model parametersθ\\theta, a planner call is given byat=π⁡\(st,θ\)a\_\{t\}=\\pi\(s\_\{t\},\\theta\), where we overloadπ\\pifor notational convenience in the next paragraph\. Planners are often instantiated as sampling\-based algorithms, e\.g\., MPPI\([Williams et al\., 2015](https://arxiv.org/html/2609.12278#bib.bib1)\)\.

We refer to the tupleζt=\(st,at,rt,st\+1\)\\zeta\_\{t\}=\(s\_\{t\},a\_\{t\},r\_\{t\},s\_\{t\+1\}\)as a transition\. A trajectoryτ=\(ζ0,ζ1,…,ζM−1\)\\tau=\(\\zeta\_\{0\},\\zeta\_\{1\},\\dots,\\zeta\_\{M\-1\}\)collects a sequence ofMMtransitions\.p⁡\(τ∣e,π\)p\(\\tau\\mid e,\\pi\)denotes the distribution over trajectories induced by acting in environmenteeaccording to the policy or plannerπ\\pi\. A batch of trajectories with sizeNNis the tupleb=\(τ1,…,τN\)b=\(\\tau\_\{1\},\\dots,\\tau\_\{N\}\), where each trajectory is drawn independently and identically distributed fromp⁡\(τ∣e,π\)p\(\\tau\\mid e,\\pi\)\. At test time, the agent interacts with environments fromp⁡\(ℰ\)p\(\\mathcal\{E\}\)which it may or may not have encountered during pretraining\.

### 3\.1World Model

World models are learned approximations of MDPs that agents can use to plan actions\. They often consist of an encoder, a dynamics function, a value function, and a reward function\([Hafner et al\., 2019](https://arxiv.org/html/2609.12278#bib.bib12);[Hafner et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib13);[Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14);[Levy et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib26)\)\. These modules are neural networks jointly trained using data from high\-fidelity simulation or the real world\. We refer to the parameters for all modules asθw\\theta\_\{w\}, thebaseparameters\. In this paper, the agent usesθw\\theta\_\{w\}with a plannerπupd\\pi\_\{\\mathrm\{upd\}\}to select actions, i\.e\.,at=πupd​\(st,θw\)a\_\{t\}=\\pi\_\{\\mathrm\{upd\}\}\(s\_\{t\},\\theta\_\{w\}\)\.[Figure1](https://arxiv.org/html/2609.12278#S3.F1)shows the control flow for action selection given a set of candidate actions\.

EncoderDynamicsRewardValuests\_\{t\}ztz\_\{t\}ata\_\{t\}zt\+1z\_\{t\+1\}rt\+1r\_\{t\+1\}vt\+1v\_\{t\+1\}

Figure 1:A single forward pass of a world model used to plan by maximizing predicted return\. Blocks in red have trainable parameters, which constituteθw\\theta\_\{w\}\.First, the encoder produces a latent stateztz\_\{t\}from an environment statests\_\{t\}\. Then, the dynamics module predicts the next latent statezt\+1z\_\{t\+1\}givenztz\_\{t\}andata\_\{t\}\. Next, the reward and value modules usezt\+1z\_\{t\+1\}to compute the rewardrt\+1r\_\{t\+1\}and valuevt\+1v\_\{t\+1\}\. Finally, the planner ranksata\_\{t\}using the estimated returnrt\+1\+γ​vt\+1r\_\{t\+1\}\+\\gamma\\,v\_\{t\+1\}, whereγ∈\[0,1\)\\gamma\\in\[0,1\)is the discount factor\.

In this paper, we apply our method,CLAW, to the TD\-MPC2 world model architecture\([Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14)\)because it consistently achieves strong performance across a variety of continuous control tasks using a single set of hyperparameters\.111TD\-MPC2 pretrains a model\-free policy to seed a sampling\-based planner\. To avoid confusion, we include the parameters for this policy withinθw\\theta\_\{w\}\.We emphasize thatCLAWis agnostic to world model architecture since our only requirement is that we can sample environments during pretraining and insert LoRA adapters into world model modules\. Therefore,CLAWcan be applied to, e\.g\., action\-conditioned JEPA world models\([Maes et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib54);[Assran et al\., 2025](https://arxiv.org/html/2609.12278#bib.bib55)\)or Dreamer\-style architectures\([Hafner et al\., 2019](https://arxiv.org/html/2609.12278#bib.bib12);[Hafner et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib13)\)\.

### 3\.2Low\-Rank Adaptation

At test time, the base parametersθw\\theta\_\{w\}stay frozen and we insert LoRA adapters\([Hu et al\., 2021](https://arxiv.org/html/2609.12278#bib.bib16)\)into the world model’s layers\. Concretely, for an adapted layer with base weight matrixW0∈ℝdout×dinW\_\{0\}\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\_\{\\mathrm\{in\}\}\}, wheredoutd\_\{\\mathrm\{out\}\}anddind\_\{\\mathrm\{in\}\}are the layer’s output and input dimensions, the effective weight is

W=W0\+A​B,A∈ℝdout×dr,B∈ℝdr×din,dr≪min⁡\(dout,din\)\.W\\;=\\;W\_\{0\}\\;\+\\;AB,\\qquad A\\in\\mathbb\{R\}^\{d\_\{\\mathrm\{out\}\}\\times d\_\{r\}\},\\qquad B\\in\\mathbb\{R\}^\{d\_\{r\}\\times d\_\{\\mathrm\{in\}\}\},\\qquad d\_\{r\}\\ll\\min\(d\_\{\\mathrm\{out\}\},d\_\{\\mathrm\{in\}\}\)\.\(1\)The factorsAAandBBconstitute the adapter, anddrd\_\{r\}is the low\-rank bottleneck dimension\. We defineϕw\\phi\_\{w\}as the collection of allA,BA,Bfactors across the adapted layers\. For simplicity, we adapt all world model layers and keep rankdrd\_\{r\}constant\.

### 3\.3Hypernetworks

A hypernetwork is a neural networkHθhH\_\{\\theta\_\{h\}\}that outputs the weights of another network\([Ha et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib10)\)\. Given an inputxx, the target network parametersϕ\\phiare produced as

ϕ=Hθh​\(x\),\\phi=H\_\{\\theta\_\{h\}\}\(x\),\(2\)whereθh\\theta\_\{h\}are the weights for the hypernetwork\. In the next section, we present more details on the architecture and training procedure for our LoRA\-generating hypernetwork\.

## 4Approach

We now presentCLAW, a model\-based RL adaptation algorithm that uses a hypernetwork to generate low\-rank adapters at test time\. This section is structured as follows: First, we detail the architecture of our hypernetwork \([Section4\.1](https://arxiv.org/html/2609.12278#S4.SS1)\)\. Then, we describe a pretraining procedure to jointly train the base model and hypernetwork \([Section4\.2](https://arxiv.org/html/2609.12278#S4.SS2)\)\.

### 4\.1Hypernet\-Based Adapter Generation

hembh\_\{\\mathrm\{emb\}\}hembh\_\{\\mathrm\{emb\}\}mean pool hconh\_\{\\mathrm\{con\}\}ζ1\\zeta\_\{1\}ζM\\zeta\_\{M\}bconb\_\{\\mathrm\{con\}\}uembu\_\{\\mathrm\{emb\}\}hgenh\_\{\\mathrm\{gen\}\}uconu\_\{\\mathrm\{con\}\}contextembeddinguku\_\{k\}chunkembedding11kkKKΔ​ϕw\\Delta\\phi\_\{w\}\+\+ϕw\\phi\_\{w\}Δ​ϕw\\Delta\\phi\_\{w\}ϕ0\\phi\_\{0\}template

Figure 2:Forward pass of the hypernetworkHθhH\_\{\\theta\_\{h\}\}when generating chunkkkofΔ​ϕw\\Delta\\phi\_\{w\}using a batchbconb\_\{\\mathrm\{con\}\}ofNconN\_\{\\mathrm\{con\}\}trajectories, each withMMtransitions\. In thebconb\_\{\\mathrm\{con\}\}block, depth represents the batch dimension\. Blocks in red have trainable parameters which constituteθh\\theta\_\{h\}\.We use a hypernetwork to generate LoRA adapters for each environment, online at test time\. Using a context batchbconb\_\{\\mathrm\{con\}\}ofNconN\_\{\\mathrm\{con\}\}trajectories ofMMtransitions each, it generates LoRA adaptersϕw\\phi\_\{w\}for the world model’s base weightsθw\\theta\_\{w\}\. Formally:

ϕw=Hθh​\(bcon\)\.\\phi\_\{w\}\\;=\\;H\_\{\\theta\_\{h\}\}\(b\_\{\\mathrm\{con\}\}\)\.\(3\)We now describe the architecture ofHHand howbconb\_\{\\mathrm\{con\}\}is used to generateϕw\\phi\_\{w\}\.

First, we generate an embeddinguembu\_\{\\mathrm\{emb\}\}for each transitionζ∈bcon\\zeta\\in b\_\{\\mathrm\{con\}\}using a small multi\-layer perceptron \(MLP\)hembh\_\{\\mathrm\{emb\}\}\. Then, we average the embeddings and feed them into a second MLPhconh\_\{\\mathrm\{con\}\}to produce a context embeddinguconu\_\{\\mathrm\{con\}\}that identifies the environment\. Note that embedding each transition independently, then averaging, discards information about transition order\.

Once we have identified the environment withuconu\_\{\\mathrm\{con\}\}, we feed it intohgenh\_\{\\mathrm\{gen\}\}, an MLP, to generateΔ​ϕw\\Delta\\phi\_\{w\}, which is added to a learned templateϕ0\\phi\_\{0\}to produceϕw\\phi\_\{w\}, i\.e\.,

ϕw=ϕ0\+Δ​ϕw\.\\phi\_\{w\}=\\phi\_\{0\}\+\\Delta\\phi\_\{w\}\.\(4\)Templates have been shown to improve learning performance in hypernetworks\([Beck et al\., 2023](https://arxiv.org/html/2609.12278#bib.bib57)\)\.

GeneratingΔ​ϕw\\Delta\\phi\_\{w\}from a single forward pass would require an impractically large generator networkhgenh\_\{\\mathrm\{gen\}\}\. Instead,hgenh\_\{\\mathrm\{gen\}\}is a small network we reuse to generateΔ​ϕw\\Delta\\phi\_\{w\}in chunks\. Concretely, following[Von Oswald et al\. \(2020\)](https://arxiv.org/html/2609.12278#bib.bib56), we divideΔ​ϕw\\Delta\\phi\_\{w\}intoKKchunks and generate chunkkkby feeding the chunk embeddinguku\_\{k\}and the context embeddinguconu\_\{\\mathrm\{con\}\}intohgenh\_\{\\mathrm\{gen\}\}\.[Figure2](https://arxiv.org/html/2609.12278#S4.F2)illustrates the full procedure\.

### 4\.2Amortizing Low\-Rank Adaptation

We now describe the procedure to jointly pretrain the world model and hypernetwork, and thereby amortize test\-time low\-rank adaptation\. At a high level,CLAWwraps a standard world model pretraining algorithm with a loop that iterates between environments sampled from an environment family\. This allows us to train the hypernetwork to generate adapters for each environment\. In the following subsections we explain:1\)a single\-environment training step for the base model and hypernetwork parameters \(i\.e\.,θw\\theta\_\{w\},θh\\theta\_\{h\}\),2\)a training loss that averages over environments, thus meta\-training the base and hypernetwork for online adaptation, and3\)the full pretraining algorithm, instantiated for the TD\-MPC2 architecture\([Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14)\)\.

#### Single\-Environment Training Step\.

CLAWPlannerπupd\\pi\_\{\\mathrm\{upd\}\}Environmentℬupdj\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{j\}Lossfunctionℒ\\mathcal\{L\}θw\\theta\_\{w\}Data collectionpolicyπcon\\pi\_\{\\mathrm\{con\}\}Contextℬconj\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{j\}HypernetHθhH\_\{\\theta\_\{h\}\}θh\\theta\_\{h\}ℒ⁡\(θw,ϕw,bupd\)\\mathcal\{L\}\(\\theta\_\{w\},\\phi\_\{w\},b\_\{\\mathrm\{upd\}\}\)bupdb\_\{\\mathrm\{upd\}\}bconb\_\{\\mathrm\{con\}\}ϕw\\phi\_\{w\}LoRAadapters

Figure 3:Computing the lossℒ\\mathcal\{L\}for environmentjjduringCLAWpretraining\. Green marks whatCLAWadds to standard world model pretraining loss computation\. Red marks trainable parameters and their gradient path, and ticks mark a stopped gradient\.CLAWcan be applied to any world model training algorithm with the same high\-level computation graph\.Given environmentjj, we start by filling two buffers:ℬconj\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{j\}, a context buffer containing trajectories collected using a context\-collection policyπcon\\pi\_\{\\mathrm\{con\}\}, andℬupdj\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{j\}, a buffer containing trajectories collected using the agent’s planner,πupd\\pi\_\{\\mathrm\{upd\}\}\. Next, we sample a batchbconb\_\{\\mathrm\{con\}\}ofNconN\_\{\\mathrm\{con\}\}trajectories fromℬconj\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{j\}and produce environment\-specific low\-rank adapters using the hypernetwork, i\.e\.,ϕw=Hθh​\(bcon\)\\phi\_\{w\}=H\_\{\\theta\_\{h\}\}\(b\_\{\\mathrm\{con\}\}\)\. Then, we combineθw\\theta\_\{w\}andϕw\\phi\_\{w\}per[Eq\.1](https://arxiv.org/html/2609.12278#S3.E1), and use a batchbupdb\_\{\\mathrm\{upd\}\}ofNupdN\_\{\\mathrm\{upd\}\}trajectories sampled fromℬupdj\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{j\}to compute the lossℒ\\mathcal\{L\}\(detailed in the next section\)\. Finally, we differentiateℒ\\mathcal\{L\}with respect toθw\\theta\_\{w\}andθh\\theta\_\{h\}, and updateθw\\theta\_\{w\}andθh\\theta\_\{h\}with an Adam optimizer\([Kingma and Ba, 2014](https://arxiv.org/html/2609.12278#bib.bib64)\)\.

The result is a hypernetwork trained to produce low\-rank adaptersϕw\\phi\_\{w\}forθw\\theta\_\{w\}usingbconb\_\{\\mathrm\{con\}\}, and a base model trained to be adapted withϕw\\phi\_\{w\}\. A visualization of this procedure can be seen in[Fig\.3](https://arxiv.org/html/2609.12278#S4.F3)\.

Our method collectsbconb\_\{\\mathrm\{con\}\}using the context\-collection policyπcon\\pi\_\{\\mathrm\{con\}\}instead of the agent’s planner\. There are two reasons for this approach:

1. 1\.It ensures the distribution of context trajectories is the same during training and testing, facilitating environment identification for the hypernetworkHH\.
2. 2\.In future work, we can directly optimize the policyπcon\\pi\_\{\\mathrm\{con\}\}to produce trajectoriesbconb\_\{\\mathrm\{con\}\}which are better at identifying the environment and/or lead to better test\-time performance\.

In our experiments,πcon\\pi\_\{\\mathrm\{con\}\}is always a random\-action policy\. This fixes context data quality across adaptation baselines so that performance differences reflect the adaptation method rather than the collected context data\. However, practitioners may choose to use a different policy, e\.g\., one that is safe for real\-life deployment \(but not necessarily high\-performing\)\. The use of a non\-randomπcon\\pi\_\{\\mathrm\{con\}\}is fully supported byCLAWas long asπcon\\pi\_\{\\mathrm\{con\}\}can be queried during pretraining\.

#### Multi\-Environment Training Loss\.

The loss in[Fig\.3](https://arxiv.org/html/2609.12278#S4.F3)decomposes as

ℒ⁡\(θw,ϕw,bupd\)=ℒTDMPC2​\(Θw,bupd\)\+λ​‖ϕw‖22,\\mathcal\{L\}\(\\theta\_\{w\},\\phi\_\{w\},b\_\{\\mathrm\{upd\}\}\)\\;=\\;\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\}\}\(\\Theta\_\{w\},b\_\{\\mathrm\{upd\}\}\)\\;\+\\;\\lambda\\\|\\phi\_\{w\}\\\|\_\{2\}^\{2\},\(5\)whereΘw\\Theta\_\{w\}are the merged base and adapter weights per[Eq\.1](https://arxiv.org/html/2609.12278#S3.E1),λ∈ℝ≥0\\lambda\\in\\mathbb\{R\}\_\{\\geq 0\}weighs the adapter’sL2L\_\{2\}penalty, andℒTDMPC2\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\}\}is the training loss for a TDMPC2 world model described by[Hansen et al\. \(2024\)](https://arxiv.org/html/2609.12278#bib.bib14)\. If a different world model architecture is used,ℒTDMPC2\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\}\}may be replaced with the corresponding loss\.[Figure3](https://arxiv.org/html/2609.12278#S4.F3)computes a single\-environment, single\-batch approximation of the objective in the following minimization problem:

minθw,θh⁡𝔼e​\[𝔼bcon,bupd​\[ℒ\]\],\\min\_\{\\theta\_\{w\},\\,\\theta\_\{h\}\}\\;\\mathbb\{E\}\_\{e\}\\left\[\\,\\mathbb\{E\}\_\{b\_\{\\mathrm\{con\}\},\\,b\_\{\\mathrm\{upd\}\}\}\\left\[\\mathcal\{L\}\\right\]\\right\],\(6\)wheree∼p⁡\(ℰ\)e\\sim p\(\\mathcal\{E\}\), andbconb\_\{\\mathrm\{con\}\}andbupdb\_\{\\mathrm\{upd\}\}are batches ofNconN\_\{\\mathrm\{con\}\}andNupdN\_\{\\mathrm\{upd\}\}trajectories, respectively\. Each trajectory is drawn i\.i\.d\. fromp⁡\(τ∣e,πcon\)p\(\\tau\\mid e,\\pi\_\{\\mathrm\{con\}\}\)andp⁡\(τ∣e,πupd\)p\(\\tau\\mid e,\\pi\_\{\\mathrm\{upd\}\}\), respectively\.

1:Environment family

\{E1,…,En\}\\\{E^\{1\},\\,\\ldots,\\,E^\{n\}\\\}, context\-collection policy

πcon\\pi\_\{\\mathrm\{con\}\}, initial

θw,θh\\theta\_\{w\},\\,\\theta\_\{h\}, total training steps

TT
2:

3:// Context data collection

4:forj=1,…,nj=1,\\ldots,ndo

5:Init\. and fillℬconj\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{j\}by acting inEjE^\{j\}withπcon\\pi\_\{\\mathrm\{con\}\}

6:

7:// Update data prefill

8:forj=1,…,nj=1,\\ldots,ndo

9:Init\. and fill

ℬupdj\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{j\}by acting in

EjE^\{j\}with

πupd\\pi\_\{\\mathrm\{upd\}\}
10:

11:// Online Interaction and Training Loop

12:i∼Uniform⁡\(\{1,…,n\}\)i\\;\\sim\\;\\mathrm\{Uniform\}\(\\\{1,\\ldots,n\\\}\)⊳\\trianglerightSample rollout env\.

13:

s←Ei\.reset⁡\(\)s\\;\\leftarrow\\;E^\{i\}\.\\mathrm\{reset\}\(\)
14:for

t=1,…,Tt=1,\\ldots,Tdo

15:k∼Uniform⁡\(\{1,…,n\}\)k\\;\\sim\\;\\mathrm\{Uniform\}\(\\\{1,\\ldots,n\\\}\)⊳\\trianglerightSample training env\.

16:

17:// Update base and hypernet parameters

18:

bupd←Sample⁡\(ℬupdk\)b\_\{\\mathrm\{upd\}\}\\;\\leftarrow\\;\\mathrm\{Sample\}\(\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{k\}\)
19:bcon←Sample⁡\(ℬconk\)b\_\{\\mathrm\{con\}\}\\;\\leftarrow\\;\\mathrm\{Sample\}\(\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{k\}\)

20:ϕw←Hθh​\(bcon\)\\phi\_\{w\}\\;\\leftarrow\\;H\_\{\\theta\_\{h\}\}\(b\_\{\\mathrm\{con\}\}\)⊳\\trianglerightTraining env\. adapters

21:Θw←\(θw,ϕw\)\\Theta\_\{w\}\\;\\leftarrow\\;\(\\theta\_\{w\},\\;\\phi\_\{w\}\)⊳\\trianglerightMerge adapters

22:

\(θw,θh\)←TDMPC2Update​\(bupd,bcon,Θw,θh\)\\left\(\\theta\_\{w\},\{\\hbox\{\\pagecolor\{added\}$\\theta\_\{h\}$\}\}\\right\)\\leftarrow\\textsc\{TDMPC2Update\}\(b\_\{\\mathrm\{upd\}\},\{\\hbox\{\\pagecolor\{added\}$b\_\{\\mathrm\{con\}\}$\}\},\\Theta\_\{w\},\{\\hbox\{\\pagecolor\{added\}$\\theta\_\{h\}$\}\}\)
23:

24:// Plan and act

25:bconi←Sample⁡\(ℬconi\)b\_\{\\mathrm\{con\}\}^\{i\}\\;\\leftarrow\\;\\mathrm\{Sample\}\(\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{i\}\)

26:ϕw←Hθh​\(bconi\)\\phi\_\{w\}\\;\\leftarrow\\;H\_\{\\theta\_\{h\}\}\(b\_\{\\mathrm\{con\}\}^\{i\}\)⊳\\trianglerightRollout env\. adapters

27:Θw←\(θw,ϕw\)\\Theta\_\{w\}\\;\\leftarrow\\;\(\\theta\_\{w\},\\,\\phi\_\{w\}\)

28:

a←πupd​\(s,Θw\)a\\;\\leftarrow\\;\\pi\_\{\\mathrm\{upd\}\}\{\(s,\\;\\Theta\_\{w\}\)\}⊳\\trianglerightPlan with latest parameters

29:

s′,r,done←Ei\.step⁡\(a\)s^\{\\prime\},\\;r,\\;\\mathrm\{done\}\\;\\leftarrow\\;E^\{i\}\.\\mathrm\{step\}\\\!\\left\(a\\right\)
30:

31:// Update environment and buffers

32:

ℬupdi←ℬupdi∪\{\(s,a,r,s′\)\}\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{i\}\\;\\leftarrow\\;\\mathcal\{B\}\_\{\\mathrm\{upd\}\}^\{i\}\\cup\\\{\(s,\\;a,\\;r,\\;s^\{\\prime\}\)\\\}
33:if

done\\mathrm\{done\}then

34:i∼Uniform⁡\(\{1,…,n\}\)i\\;\\sim\\;\\mathrm\{Uniform\}\(\\\{1,\\ldots,n\\\}\)⊳\\trianglerightSample rollout env\.

35:

s←Ei\.reset⁡\(\)s\\;\\leftarrow\\;E^\{i\}\.\\mathrm\{reset\}\(\)
36:else

37:

s←s′s\\;\\leftarrow\\;s^\{\\prime\}
38:

θw,θh\\theta\_\{w\},\\,\\theta\_\{h\}

Algorithm 1CLAW\+ TD\-MPC2 pretrainingProblem[Equation6](https://arxiv.org/html/2609.12278#S4.E6)illustrates how our method can be applied to any world model architecture\. If we removebconb\_\{\\mathrm\{con\}\}and theλ​‖ϕw‖22\\lambda\\\|\\phi\_\{w\}\\\|\_\{2\}^\{2\}term, the inner expectation of[Eq\.6](https://arxiv.org/html/2609.12278#S4.E6)is the objective of a standard world model training problem\. Our approach,CLAW, defines the hypernetwork machinery \(green box in[Fig\.3](https://arxiv.org/html/2609.12278#S4.F3)\), and uses it to amortize low\-rank adaptation by wrapping the standard objective with an expectation over the environment familyp⁡\(ℰ\)p\(\\mathcal\{E\}\)\.

#### ApplyingCLAWto the TD\-MPC2 Architecture\.

[Algorithm1](https://arxiv.org/html/2609.12278#alg1)shows the TD\-MPC2\([Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14)\)pretraining algorithm modified to useCLAW\. It includes the following additions:1\)logic to sample a rollout environment fromp⁡\(ℰ\)p\(\\mathcal\{E\}\)every episode, thereby approximating the outer expectation in[Eq\.6](https://arxiv.org/html/2609.12278#S4.E6);2\)logic to sample atrainingenvironment fromp⁡\(ℰ\)p\(\\mathcal\{E\}\)every step, thereby stabilizing multi\-environment training;3\)a bufferℬconj\\mathcal\{B\}\_\{\\mathrm\{con\}\}^\{j\}per environmentjjwith corresponding samplesbconb\_\{\\mathrm\{con\}\}, and4\)a forward pass ofHθhH\_\{\\theta\_\{h\}\}to produce adaptersϕw\\phi\_\{w\}givenbconb\_\{\\mathrm\{con\}\}\.

We denote additions to the original TD\-MPC2 algorithm withgreen highlightsin[Algorithm1](https://arxiv.org/html/2609.12278#alg1)\. Our algorithm uses a modified version of the TD\-MPC2 weight update,TDMPC2Update, which we include as[Algorithm2](https://arxiv.org/html/2609.12278#alg2)in[AppendixA](https://arxiv.org/html/2609.12278#A1)\. The result of[Algorithm1](https://arxiv.org/html/2609.12278#alg1)is a base TD\-MPC2 model and hypernet that can generate test\-time LoRA adapters for any layer of the base model\.

We emphasize thatCLAWcan be applied regardless of the chosen world model architecture or training algorithm\. Specifically, an adapter\-generating hypernetwork can be trained as long as one can sample environments, maintain a buffer for each of them, and maintain an unbroken gradient chain between the loss and hypernetwork \(as shown by the red lines in[Fig\.3](https://arxiv.org/html/2609.12278#S4.F3)\)\.

## 5Experiments

We testCLAWon the environment families shown in[Table1](https://arxiv.org/html/2609.12278#S5.T1), where each family consists of a set of environments that differ in one property\. We use 5 seeds per experiment and report 95% confidence intervals for the mean across seeds, computed with Student’s t distribution\.[AppendixB](https://arxiv.org/html/2609.12278#A2)lists the architecture, optimization, and per\-experiment hyperparameters needed to reproduce every experiment below\. We will include a public codebase in the final version of this paper\.

Table 1:Environment families used in our experiments, adapted from DeepMind Control Suite\([Tassa et al\., 2018](https://arxiv.org/html/2609.12278#bib.bib65)\)and Meta\-World\([Yu et al\., 2019](https://arxiv.org/html/2609.12278#bib.bib66)\)\. The environment property column lists the property that varies between environments in a family\.### 5\.1Experiment 1: Online Adaptation

#### Question\.

How doesCLAW’s performance compare to online, gradient\-based adaptation?

#### Setup\.

For each environment in a family, we begin by collecting 3 episodes of context data withπcon\\pi\_\{\\mathrm\{con\}\}\(7\.5–75s of data depending on the environment\)\. Then, we roll out a pretrained model, adapting it at every step using the context data\. Gradient\-based baselines take one gradient step per environment step, adapting a set of LoRA adapters\. On the other hand,CLAWuses its hypernetwork to generate new LoRA adapters at every environment step\. We report episode return \(or success rate\) averaged across environments and seeds\.

#### Baselines\.

We compareCLAWagainst adaptation on models trained on a single task \(ST\) or with Domain Randomization\([Tobin et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib47), DR;\)\. DR pretrains a base model on all environments in the family, and ST pretrains a base model on a single environment in the family\. We also compare against a non\-adapting oracle baseline, which appends the environment’s ground\-truth parameter directly to the state, e\.g\., actuator strength, or running direction \(specified with a one\-hot vector\)\. Since every other method must infer this parameter from context data, the oracle’s performance serves as a ceiling on the performance achievable with perfect environment identification\. World model hyperparameters are fixed across baselines\.

#### Results\.

Results are shown in[Fig\.4](https://arxiv.org/html/2609.12278#S5.F4)\. In all families,CLAWclosely tracks the oracle’s performance\. In the manipulation families \(Push and Reach\), DR and ST climb toward that ceiling, but they take many environment steps to get there, and performance is very noisy\. In Walker, DR and ST get worse instead of catching up\. In Cheetah, ST catches up but DR does not\. DR’s low performance is likely due to Cheetah’s two running directions forming a bimodal distribution of behavior, which DR training resolves into a single “average” model that is not good at either behavior\.

![Refer to caption](https://arxiv.org/html/2609.12278v1/figures/exp1_all_families.png)Figure 4:Experiment 1\. Episode return \(or success rate\) vs\. test\-time environment steps, averaged over all environments in a family\.

### 5\.2Experiment 2: Adaptation when Data Is Limited

#### Question\.

Given that test\-time data is often expensive and/or scarce, how doesCLAW’s performance compare to the return achieved by applying many steps of gradient\-based adaptation on a fixed test\-time dataset?

#### Setup\.

For each family and environment, we collect 3 episodes of context data usingπcon\\pi\_\{\\mathrm\{con\}\}\(between 7\.5 and 75 seconds depending on the environment\)\. Then, we use this data to adapt all methods offline, freeze the models, and evaluate them\. For the gradient\-based adaptation baselines, we sweep a range of offline gradient steps, thereby testing how performance changes as a function of offline adaptation\. We report the same metrics as Experiment 1\.

#### Baselines\.

As in Experiment 1, we compare to DR, ST, and an oracle, here also swept over adaptation gradient steps\.

#### Results\.

Results are shown in[Fig\.5](https://arxiv.org/html/2609.12278#S5.F5)\.CLAWmostly tracks oracle performance, and outperforms other baselines\. Moreover, as the number of SGD steps increases, performance collapses for all gradient\-based methods as they overfit to the data\. This is supported by the collapse of the oracle: since it already knows the environment, the only thing left for it to fit is noise in the small dataset\.

![Refer to caption](https://arxiv.org/html/2609.12278v1/figures/exp2_all_tasks.png)Figure 5:Experiment 2\.CLAW’s single hypernetwork pass \(red\) vs\. gradient\-based baselines \(DR, ST, oracle\) as a function of SGD steps\.

### 5\.3Experiment 3: Comparison to In\-Context Learning

![Refer to caption](https://arxiv.org/html/2609.12278v1/figures/exp3_sweep_eval_reward.png)Figure 6:Experiment 3\.Left:return vs\. model size\.Right:reward relative toCLAW\.#### Question\.

DoesCLAW’s performance depend on generating expressive world\-model adapters, or merely conditioning on the context embedding? How does this relationship change with model size?

#### Setup\.

For each environment in the Cheetah/Walker family in[Table1](https://arxiv.org/html/2609.12278#S5.T1), we roll out one episode adapting at every timestep \(as in previous experiments\)\. We choose this family because we need a harder family than the others to tell the in\-context baselines apart\. Cheetah/Walker provides this since state and action spaces differ completely across environments\. We also sweep over model size and report the same evaluation metrics as Experiment 1\.

#### Baselines\.

We compare against two in\-context learning baselines that condition a fixed model on context, rather than adapting its weights, to isolate the source ofCLAW’s performance\. Concatenation \(Concat\) feeds the hypernetwork’s context embedding directly into the world model’s state \(instead of using it to generate adapters\); this is akin to the approaches of[Kumar et al\. \(2021\)](https://arxiv.org/html/2609.12278#bib.bib23);[Rakelly et al\. \(2019\)](https://arxiv.org/html/2609.12278#bib.bib36);[Liu et al\. \(2025\)](https://arxiv.org/html/2609.12278#bib.bib27)\. We also compare to a hypernetwork trained to generate FiLM parameters\([Perez et al\., 2017](https://arxiv.org/html/2609.12278#bib.bib32)\), which scale and shift features\.

#### Results\.

Results are shown in[Fig\.6](https://arxiv.org/html/2609.12278#S5.F6)\. At the smallest sizes, no method learns enough to separate performance\. At the largest sizes, the base model has enough capacity to represent every environment without conditioning, so performance does not separate either\.CLAW’s advantage is clearest at intermediate sizes where it outperforms both baselines\. These results align with those of[Jayakumar et al\. \(2020\)](https://arxiv.org/html/2609.12278#bib.bib19), who predict that hypernetwork\-generated weights are more expressive than the conditioning mechanisms of FiLM and Concat\.

### 5\.4Experiment 4: LoRA Rank Sensitivity

#### Question\.

For a fixed model size, how doesCLAW’s performance change with LoRA rank?

#### Setup\.

We fix the model size and pretrain a hypernetwork withCLAWfor a range of LoRA ranks\. We use the Cheetah/Walker family and select an intermediate size where in\-context baseline performance is separable\. As LoRA rank increases, we increase the number of chunk embeddingsKKin[Fig\.2](https://arxiv.org/html/2609.12278#S4.F2)and keep all other hyperparameters fixed\. This controls for hypernetwork capacity, so any performance change reflects the rank increase, not a larger generator network\. Then, we evaluate online adaptation performance as in Experiment 1\.

![Refer to caption](https://arxiv.org/html/2609.12278v1/figures/exp4_rank_paramfrac_stacked.png)Figure 7:Experiment 4\.Top:return vs\. LoRA rank\.Bottom:return vs\. adapter size as a percentage of the base parameter count\.
#### Baselines\.

We compare against the same in\-context learning baselines as Experiment 3\.

#### Results\.

Results are shown in[Fig\.7](https://arxiv.org/html/2609.12278#S5.F7)\. We report performance vs\. adapter rank and adapter size as a percentage of the base parameter count\.CLAW’s performance rises with rank and percentage\. On the parameter axis, FiLM outperformsCLAWat low budgets, andCLAWneeds around four times FiLM’s parameter budget to match its return\. Past that budget,CLAWsurpasses FiLM, whose performance stays constant since it has no comparable knob to spend a larger budget on\.

### 5\.5Experiment 5: Post\-Hoc Hypernet Comparison

#### Question\.

How does training the base model and hypernetwork jointly \(CLAW\) compare to training a hypernetwork on top of a frozen pretrained base model\([Bianchi et al\., 2026](https://arxiv.org/html/2609.12278#bib.bib4), cf\.\)?

![Refer to caption](https://arxiv.org/html/2609.12278v1/figures/exp5_posthoc_reward.png)Figure 8:Experiment 5\. Cheetah return forCLAWvs\. post\-hoc baseline\.
#### Setup\.

We evaluate performance on Cheetah using the same metrics as previous experiments\. The post\-hoc hypernet’s total number of training steps and training data match whatCLAWwould see for this family \(500k steps total, split between both environments\)\.

#### Baseline\.

First, we pretrain a plain TD\-MPC2 base on Walker only for 250k steps\. Then, we freeze it and train a hypernetwork for 250k steps to generate adapters for the Cheetah environment\. We refer to this as the “post\-hoc hypernet\.” This matches the approach proposed by[Bianchi et al\. \(2026\)](https://arxiv.org/html/2609.12278#bib.bib4), who train a hypernetwork to generate LoRA adapters using a frozen robotics foundation model\.

#### Results\.

Results are shown in[Fig\.8](https://arxiv.org/html/2609.12278#S5.F8)\.CLAWoutperforms a post\-hoc hypernet, showing that jointly training the base model and hypernetwork leads to better performance\.

## 6Conclusion, Limitations, and Future Work

We introduceCLAW, a method for world model adaptation that uses a hypernetwork to generate LoRA adapters online, replacing gradient\-based test\-time adaptation\. Across all tested environment families,CLAWoutperforms online gradient\-based and in\-context learning baselines\. Moreover, under limited data,CLAWavoids the overfitting that degrades gradient\-based approaches\. We also present ablations showing thatCLAW’s performance comes from generating world model adapters rather than merely conditioning on a context embedding, that performance improves with LoRA rank, and that training the hypernetwork jointly with the base model outperforms training it post hoc\.

CLAWis limited by the identifiability of the environment when using the data\-collection policyπcon\\pi\_\{\\mathrm\{con\}\}\. Additionally,CLAWlearns to generate adapters only for the environments seen during pretraining\. Future work includes pretraining and/or architectural improvements for better generalization, applyingCLAWto sim\-to\-real transfer, optimizing the data\-collection policy for downstream performance, and algorithmically selecting which environments to train on\([Dennis et al\., 2020](https://arxiv.org/html/2609.12278#bib.bib67)\)\.

## AI Use Statement

We used generative AI to implement methods and for general coding assistance, to edit and format this paper, and to summarize existing literature and surface relevant work we had not considered\. We did not use generative AI for any other task\. We have reviewed all AI\-assisted work\. We take responsibility for the final content of this work, including text, claims or artifacts produced with the aid of generative AI\.

## Ethics Statement

We are not aware of any ethical concerns raised by this work\.

## Reproducibility Statement

[AppendixB](https://arxiv.org/html/2609.12278#A2)reports every hyperparameter used to reproduce our experiments\. We will also include a code base with instructions on how to run every experiment in this paper\.

## Appendix ATDMPC2Update: World Model and Policy Update

[Algorithm2](https://arxiv.org/html/2609.12278#alg2)is the TD\-MPC2 weight update\([Hansen et al\., 2024](https://arxiv.org/html/2609.12278#bib.bib14)\)modified to train the hypernetwork alongside the base model\. It runs in two phases\. The first phase computes the world model loss and updates the world model base sliceθwwm\\theta\_\{w\}^\{\\mathrm\{wm\}\}jointly with the hypernetwork parametersθh\\theta\_\{h\}\. The second phase computes the policy loss and updates the policy base sliceθwπ\\theta\_\{w\}^\{\\pi\}, again jointly withθh\\theta\_\{h\}\. Between the two phases we regenerate the policy adapterϕwπ\{\\phi\_\{w\}^\{\\pi\}\}with the updated hypernetwork, so the policy loss uses the adapter the current hypernetwork produces\. The update ends by refreshing the target critic and theQQ\-scale\. The target critic EMA tracks base weights only, since its adapter is regenerated on every call\.

Algorithm 2TDMPC2Update— world model, policy, and hypernetwork update1:

bupd,bconb\_\{\\mathrm\{upd\}\},\\,b\_\{\\mathrm\{con\}\},

Θw=\(θw,ϕw\)\\Theta\_\{w\}=\(\\theta\_\{w\},\\,\\phi\_\{w\}\), where

ϕw=\(ϕwwm,ϕwπ\)\\phi\_\{w\}=\(\\phi\_\{w\}^\{\\mathrm\{wm\}\},\\,\\phi\_\{w\}^\{\\pi\}\)are the world\-model and policy adapter slices,

θh\\theta\_\{h\}
2:

𝐳←encode⁡\(bupd,Θw\)\\mathbf\{z\}\\;\\leftarrow\\;\\mathrm\{encode\}\\\!\\left\(b\_\{\\mathrm\{upd\}\},\\,\\Theta\_\{w\}\\right\)
3:

ℒTDMPC2,wm←TDMPC2WorldModelLoss⁡\(bupd,𝐳,Θw\)\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\},\\,\\mathrm\{wm\}\}\\;\\leftarrow\\;\\mathrm\{TDMPC2WorldModelLoss\}\\\!\\left\(b\_\{\\mathrm\{upd\}\},\\,\\mathbf\{z\},\\,\\Theta\_\{w\}\\right\)
4:

θwwm,θwπ←θw\\theta\_\{w\}^\{\\mathrm\{wm\}\},\\,\\theta\_\{w\}^\{\\pi\}\\;\\leftarrow\\;\\theta\_\{w\}⊳\\trianglerightbreak out world\-model and policy slices

5:

\(θwwm′,θh′\)←Optimizer⁡\(\(θwwm,θh\),∇\(θwwm,θh\)ℒTDMPC2,wm\)\\left\(\{\\theta\_\{w\}^\{\\mathrm\{wm\}\}\}^\{\\prime\},\\,\{\\hbox\{\\pagecolor\{added\}$\\theta\_\{h\}^\{\\prime\}$\}\}\\right\)\\;\\leftarrow\\;\\mathrm\{Optimizer\}\\\!\\left\(\(\\theta\_\{w\}^\{\\mathrm\{wm\}\},\\,\{\\hbox\{\\pagecolor\{added\}$\\theta\_\{h\}$\}\}\),\\,\\nabla\_\{\\\!\(\\theta\_\{w\}^\{\\mathrm\{wm\}\},\\theta\_\{h\}\)\}\\,\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\},\\,\\mathrm\{wm\}\}\\right\)
6:

θw′←\(θwwm′,θwπ\)\\theta\_\{w\}^\{\\prime\}\\;\\leftarrow\\;\(\{\\theta\_\{w\}^\{\\mathrm\{wm\}\}\}^\{\\prime\},\\,\\theta\_\{w\}^\{\\pi\}\)
7:ϕwπ′←Hθh′​\(bcon\)\{\\phi\_\{w\}^\{\\pi\}\}^\{\\prime\}\\;\\leftarrow\\;H\_\{\\theta\_\{h\}^\{\\prime\}\}\(b\_\{\\mathrm\{con\}\}\)⊳\\trianglerightRefresh policy adapter

8:

Θw′←\(θw′,\(ϕwwm,ϕwπ′\)\)\\Theta\_\{w\}^\{\\prime\}\\;\\leftarrow\\;\(\\theta\_\{w\}^\{\\prime\},\\;\{\\hbox\{\\pagecolor\{added\}$\(\\phi\_\{w\}^\{\\mathrm\{wm\}\},\\,\{\\phi\_\{w\}^\{\\pi\}\}^\{\\prime\}\)$\}\}\)
9:

ℒTDMPC2,π←TDMPC2PolicyLoss⁡\(𝗌𝗀⁡\(𝐳\),Q⁡\(𝗌𝗀⁡\(Θw′\)\),Θw′\)\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\},\\,\\pi\}\\;\\leftarrow\\;\\mathrm\{TDMPC2PolicyLoss\}\\\!\\left\(\\mathsf\{sg\}\(\\mathbf\{z\}\),\\;Q\\\!\\left\(\\mathsf\{sg\}\(\\Theta\_\{w\}^\{\\prime\}\)\\right\),\\,\\Theta\_\{w\}^\{\\prime\}\\right\)
10:

\(θwπ′,θh′′\)←Optimizer⁡\(\(θwπ,θh′\),∇\(θwπ,θh′\)ℒTDMPC2,π\)\\left\(\{\\theta\_\{w\}^\{\\pi\}\}^\{\\prime\},\\,\{\\hbox\{\\pagecolor\{added\}$\\theta\_\{h\}^\{\\prime\\prime\}$\}\}\\right\)\\;\\leftarrow\\;\\mathrm\{Optimizer\}\\\!\\left\(\(\\theta\_\{w\}^\{\\pi\},\\,\{\\hbox\{\\pagecolor\{added\}$\\theta\_\{h\}^\{\\prime\}$\}\}\),\\,\\nabla\_\{\\\!\(\\theta\_\{w\}^\{\\pi\},\\theta\_\{h\}^\{\\prime\}\)\}\\,\\mathcal\{L\}\_\{\\mathrm\{TDMPC2\},\\,\\pi\}\\right\)
11:

θw′′←\(θwwm′,θwπ′\)\\theta\_\{w\}^\{\\prime\\prime\}\\;\\leftarrow\\;\(\{\\theta\_\{w\}^\{\\mathrm\{wm\}\}\}^\{\\prime\},\\,\{\\theta\_\{w\}^\{\\pi\}\}^\{\\prime\}\),

Θw′′←\(θw′′,\(ϕwwm,ϕwπ′\)\)\\Theta\_\{w\}^\{\\prime\\prime\}\\;\\leftarrow\\;\(\\theta\_\{w\}^\{\\prime\\prime\},\\;\{\\hbox\{\\pagecolor\{added\}$\(\\phi\_\{w\}^\{\\mathrm\{wm\}\},\\,\{\\phi\_\{w\}^\{\\pi\}\}^\{\\prime\}\)$\}\}\)
12:Update target critic: EMA the critic base into a persistent variable;adapter regenerated fresh, not EMA’d

13:Update

QQ\-scale

14:

θw′′,θh′′\\theta\_\{w\}^\{\\prime\\prime\},\\,\\theta\_\{h\}^\{\\prime\\prime\}

## Appendix BHyperparameters and Experimental Details

This appendix reports every hyperparameter needed to reproduce our experiments\.[Tables2](https://arxiv.org/html/2609.12278#A2.T2)and[3](https://arxiv.org/html/2609.12278#A2.T3)give the architecture, adapter, optimization, and planning hyperparameters shared by all methods and experiments unless a table below states otherwise\.[Tables4](https://arxiv.org/html/2609.12278#A2.T4),[5](https://arxiv.org/html/2609.12278#A2.T5)and[6](https://arxiv.org/html/2609.12278#A2.T6)cover Experiments 1 and 2, which share pretrained checkpoints across the Walker, Reach, Push, and Cheetah families\.[Tables7](https://arxiv.org/html/2609.12278#A2.T7),[8](https://arxiv.org/html/2609.12278#A2.T8)and[9](https://arxiv.org/html/2609.12278#A2.T9)cover Experiments 3, 4, and 5, which all use the Cheetah/Walker family\.

### B\.1Shared Architecture and Adapter Hyperparameters

Table 2:Network architecture, shared byCLAWand all baselines \(DR, ST, oracle\), and by every experiment and family unless overridden in[Tables7](https://arxiv.org/html/2609.12278#A2.T7)and[8](https://arxiv.org/html/2609.12278#A2.T8)\. The number of hypernetwork chunks isK=⌈Nadapt/c⌉K=\\lceil N\_\{\\mathrm\{adapt\}\}/c\\rceil, whereNadaptN\_\{\\mathrm\{adapt\}\}is the number of parameters inϕw\\phi\_\{w\}; it grows automatically with LoRA rank \([Section5\.4](https://arxiv.org/html/2609.12278#S5.SS4)\)\.Table 3:Optimization and planning hyperparameters, shared acrossCLAWand all baselines\.
### B\.2Experiments 1 and 2: Environment Families and Pretraining

Table 4:Per\-family pretraining setup for Experiments 1 and 2, which adapt the same checkpoints\. All methods \(CLAW, DR, ST, oracle\) pretrain for the same number of steps per family with 5 seeds each\. The oracle baseline appends the ground\-truth environment parameter to the state, increasingdim\(s\)\\dim\(s\)by 1 \(Walker actuator scale, Cheetah running direction\) or 3 \(Reach/Push goal position\)\.Table 5:Experiment 1 online adaptation protocol\. Every method rolls out for 30,000 environment steps per environment, with 5 seeds; Optimizer settings for DR/ST’s gradient step are in[Table3](https://arxiv.org/html/2609.12278#A2.T3)\.Table 6:Experiment 2 offline adaptation protocol\. All methods adapt on the same fixed context dataset, then freeze and evaluate\. Pretraining is shared with Experiment 1 \([Table4](https://arxiv.org/html/2609.12278#A2.T4)\)\.
### B\.3Experiments 3–5: Cheetah/Walker Model\-Capacity Studies

Experiments 3, 4, and 5 all use the Cheetah/Walker family \(dim\(s\)=24\\dim\(s\)\{=\}24, padded for Cheetah;dim\(a\)=6\\dim\(a\)\{=\}6; episode length 1000, 2 environments\) and share an evaluation protocol: each method re\-adapts before every planner call, evaluated over 10 episodes per environment with 5 seeds\.

Table 7:Experiment 3 model\-size sweep\. Size label C is unused\. Params\. column is the base world model parameter count\|θw\|\|\\theta\_\{w\}\|\(excludes hypernetwork parametersθh\\theta\_\{h\}\), matching the x\-axis of[Fig\.6](https://arxiv.org/html/2609.12278#S5.F6)\. All other hyperparameters follow[Tables2](https://arxiv.org/html/2609.12278#A2.T2)and[3](https://arxiv.org/html/2609.12278#A2.T3)\. Sizes F–I additionally shrink LoRA rank, tied across all five adapted modules; FiLM and Concat baselines use the same feature widths with no rank\.Table 8:Experiment 4 rank sweep\. Backbone size is fixed at F \([Table7](https://arxiv.org/html/2609.12278#A2.T7)\); rank is swept jointly across all world model parameters \(5 seeds each\) while all other hyperparameters stay fixed\. Return is mean±\\pm95% CI across 5 seeds, matching[Fig\.7](https://arxiv.org/html/2609.12278#S5.F7)\.Table 9:Experiment 5 post\-hoc comparison, at size F \([Table7](https://arxiv.org/html/2609.12278#A2.T7)\), rank 8, 5 seeds\.CLAW’s checkpoints are reused from Experiment 3, evaluated on the Cheetah task only to match the post\-hoc baseline’s single\-task evaluation\. Both methods see 250,000 steps of Cheetah data and 500,000 total training steps; the only remaining difference is joint vs\. sequential training of the base model and hypernetwork\.

## References

- A\. Amin, R\. Aniceto, A\. Balakrishna, K\. Black, K\. Conley, G\. Connors, J\. Darpinian, K\. Dhabalia, J\. DiCarlo, D\. Driess, M\. Equi, A\. Esmail, Y\. Fang, C\. Finn, C\. Glossop, T\. Godden, I\. Goryachev, L\. Groom, H\. Hancock, K\. Hausman, G\. Hussein, B\. Ichter, S\. Jakubczak, R\. Jen, T\. Jones, B\. Katz, L\. Ke, C\. Kuchi, M\. Lamb, D\. LeBlanc, S\. Levine, A\. Li\-Bell, Y\. Lu, V\. Mano, M\. Mothukuri, S\. Nair, K\. Pertsch, A\. Z\. Ren, C\. Sharma, L\. X\. Shi, L\. Smith, J\. T\. Springenberg, K\. Stachowicz, W\. Stoeckle, A\. Swerdlow, J\. Tanner, M\. Torne, Q\. Vuong, A\. Walling, H\. Wang, B\. Williams, S\. Yoo, L\. Yu, U\. Zhilinsky, and Z\. Zhouπ0\.6∗\\pi^\{\*\}\_\{0\.6\}: a VLA that learns from experience\.Note:Physical Intelligence Technical ReportExternal Links:2511\.14759Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Amos \(2023\)B\. AmosTutorial on amortized optimization\.Foundations and Trends in Machine Learning16\(5\),pp\. 592–732\.Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Andrychowiczet al\.\(2020\)O\. M\. Andrychowicz, B\. Baker, M\. Chociej, R\. Jozefowicz, B\. McGrew, J\. Pachocki, A\. Petron, M\. Plappert, G\. Powell, A\. Ray,et al\.Learning dexterous in\-hand manipulation\.The International Journal of Robotics Research39\(1\),pp\. 3–20\.Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p3.1)\.
- Assranet al\.\(2025\)M\. Assran, A\. Bardes, D\. Fan, Q\. Garrido, R\. Howes, M\. Komeili, M\. Muckley, A\. Rizvi, C\. Roberts, K\. Sinha, A\. Zholus,et al\.V\-JEPA 2: self\-supervised video models enable understanding, prediction and planning\.External Links:2506\.09985Cited by:[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p3.1)\.
- Ballet al\.\(2023\)P\. J\. Ball, L\. Smith, I\. Kostrikov, and S\. LevineEfficient online reinforcement learning with offline data\.InInternational Conference on Machine Learning,pp\. 1577–1594\.Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Becket al\.\(2023\)J\. Beck, M\. T\. Jackson, R\. Vuorio, and S\. WhitesonHypernetworks in meta\-reinforcement learning\.InConference on robot learning,pp\. 1478–1487\.Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1),[§4\.1](https://arxiv.org/html/2609.12278#S4.SS1.p3.2)\.
- Beukmanet al\.\(2023\)M\. Beukman, D\. Jarvis, R\. Klein, S\. James, and B\. RosmanDynamics generalisation in reinforcement learning via adaptive context\-aware policies\.External Links:2310\.16686Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Bianchiet al\.\(2026\)C\. Bianchi, S\. Yousefi, A\. Sampieri, A\. Roberti, L\. Rigazio, F\. Galasso, and L\. FrancoRobotic policy adaptation via weight\-space meta\-learning\.External Links:2606\.07217Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1),[§5\.5](https://arxiv.org/html/2609.12278#S5.SS5.SSS0.Px1.p1.1),[§5\.5](https://arxiv.org/html/2609.12278#S5.SS5.SSS0.Px3.p1.1)\.
- Charakornet al\.\(2025\)R\. Charakorn, E\. Cetin, Y\. Tang, and R\. T\. LangeText\-to\-lora: instant transformer adaption\.External Links:2506\.06105Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Chenet al\.\(2021\)L\. Chen, K\. Lu, A\. Rajeswaran, K\. Lee, A\. Grover, M\. Laskin, P\. Abbeel, A\. Srinivas, and I\. MordatchDecision transformer: reinforcement learning via sequence modeling\.External Links:2106\.01345Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Chenet al\.\(2024\)T\. Chen, H\. Fang, P\. Xia, X\. Liu, B\. V\. Durme, L\. Zettlemoyer, J\. Gao, and H\. ChengGenerative adapter: contextualizing language models in parameters with a single forward pass\.External Links:2411\.05877Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Davydovet al\.\(2024\)A\. Davydov, F\. Djeumou, M\. Greiff, M\. Suminaka, M\. Thompson, J\. Subosits, and T\. LewFirst, learn what you don’t know: active information gathering for driving at the limits of handling\.External Links:2411\.00107Cited by:[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1)\.
- Denniset al\.\(2020\)M\. Dennis, N\. Jaques, E\. Vinitsky, A\. Bayen, S\. Russell, A\. Critch, and S\. LevineEmergent complexity and zero\-shot transfer via unsupervised environment design\.Advances in neural information processing systems33,pp\. 13049–13061\.Cited by:[§6](https://arxiv.org/html/2609.12278#S6.p2.1)\.
- Duanet al\.\(2016\)Y\. Duan, J\. Schulman, X\. Chen, P\. L\. Bartlett, I\. Sutskever, and P\. AbbeelRL2\{\}^\{2\}: fast reinforcement learning via slow reinforcement learning\.External Links:1611\.02779Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Finnet al\.\(2017\)C\. Finn, P\. Abbeel, and S\. LevineModel\-agnostic meta\-learning for fast adaptation of deep networks\.External Links:1703\.03400Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Haet al\.\(2017\)D\. Ha, A\. Dai, and Q\. V\. LeHyperNetworks\.External Links:1609\.09106Cited by:[§1](https://arxiv.org/html/2609.12278#S1.p2.1),[§3\.3](https://arxiv.org/html/2609.12278#S3.SS3.p1.1)\.
- Ha and Schmidhuber \(2018\)D\. Ha and J\. SchmidhuberWorld models\.External Links:1803\.10122Cited by:[§1](https://arxiv.org/html/2609.12278#S1.p1.1)\.
- Hafneret al\.\(2019\)D\. Hafner, T\. Lillicrap, J\. Ba, and M\. NorouziDream to control: learning behaviors by latent imagination\.External Links:1912\.01603Cited by:[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p3.1)\.
- Hafneret al\.\(2023\)D\. Hafner, J\. Pasukonis, J\. Ba, and T\. LillicrapMastering diverse domains through world models\.External Links:2301\.04104Cited by:[§1](https://arxiv.org/html/2609.12278#S1.p1.1),[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p3.1)\.
- Hansenet al\.\(2024\)N\. Hansen, H\. Su, and X\. WangTD\-mpc2: scalable, robust world models for continuous control\.External Links:2310\.16828Cited by:[Appendix A](https://arxiv.org/html/2609.12278#A1.p1.1),[§1](https://arxiv.org/html/2609.12278#S1.p1.1),[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p1.1),[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p3.1),[§4\.2](https://arxiv.org/html/2609.12278#S4.SS2.SSS0.Px2.p1.2),[§4\.2](https://arxiv.org/html/2609.12278#S4.SS2.SSS0.Px3.p1.1),[§4\.2](https://arxiv.org/html/2609.12278#S4.SS2.p1.1)\.
- Harrisonet al\.\(2018\)J\. Harrison, A\. Sharma, and M\. PavoneMeta\-learning priors for efficient online bayesian regression\.InInternational Workshop on the Algorithmic Foundations of Robotics,pp\. 318–337\.Cited by:[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1)\.
- Huet al\.\(2021\)E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.External Links:2106\.09685Cited by:[§1](https://arxiv.org/html/2609.12278#S1.p2.1),[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p4.1),[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p2.1),[§3\.2](https://arxiv.org/html/2609.12278#S3.SS2.p1.1)\.
- Huanget al\.\(2020\)Y\. Huang, K\. Xie, H\. Bharadhwaj, and F\. ShkurtiContinual model\-based reinforcement learning with hypernetworks\.External Links:2009\.11997Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Jameset al\.\(2018\)S\. James, M\. Bloesch, and A\. J\. DavisonTask\-embedded control networks for few\-shot imitation learning\.External Links:1810\.03237Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Jayakumaret al\.\(2020\)S\. M\. Jayakumar, W\. M\. Czarnecki, J\. Menick, J\. Schwarz, J\. Rae, S\. Osindero, Y\. W\. Teh, T\. Harley, and R\. PascanuMultiplicative interactions and where to find them\.InInternational Conference on Learning Representations,Cited by:[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1),[§5\.3](https://arxiv.org/html/2609.12278#S5.SS3.SSS0.Px4.p1.1)\.
- Jianget al\.\(2022\)Y\. Jiang, A\. Gupta, Z\. Zhang, G\. Wang, Y\. Dou, Y\. Chen, L\. Fei\-Fei, A\. Anandkumar, Y\. Zhu, and L\. FanVIMA: general robot manipulation with multimodal prompts\.External Links:2210\.03094Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Jukićet al\.\(2025\)J\. Jukić, M\. Tutek, and J\. ŠnajderContext parametrization with compositional adapters\.External Links:2509\.22158Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Kimet al\.\(2026\)M\. J\. Kim, Y\. Gao, T\. Lin, Y\. Lin, Y\. Ge, G\. Lam, P\. Liang, S\. Song, M\. Liu, C\. Finn, and J\. GuCosmos policy: fine\-tuning video models for visuomotor control and planning\.External Links:2601\.16163Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Kingma and Ba \(2014\)D\. P\. Kingma and J\. BaAdam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980\.Cited by:[Table 3](https://arxiv.org/html/2609.12278#A2.T3.2.3.2),[§4\.2](https://arxiv.org/html/2609.12278#S4.SS2.SSS0.Px1.p1.1)\.
- Kostrikovet al\.\(2021\)I\. Kostrikov, A\. Nair, and S\. LevineOffline reinforcement learning with implicit q\-learning\.arXiv preprint arXiv:2110\.06169\.Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Kumaret al\.\(2021\)A\. Kumar, Z\. Fu, D\. Pathak, and J\. MalikRMA: rapid motor adaptation for legged robots\.External Links:2107\.04034Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1),[§5\.3](https://arxiv.org/html/2609.12278#S5.SS3.SSS0.Px3.p1.1)\.
- Lanieret al\.\(2025\)J\. Lanier, K\. Kim, A\. Karamzade, Y\. Liu, A\. Sinha, K\. He, D\. Corsi, and R\. FoxAdapting world models with latent\-state dynamics residuals\.External Links:2504\.02252Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Laskinet al\.\(2022\)M\. Laskin, L\. Wang, J\. Oh, E\. Parisotto, S\. Spencer, R\. Steigerwald, D\. Strouse, S\. Hansen, A\. Filos, E\. Brooks, M\. Gazeau, H\. Sahni, S\. Singh, and V\. MnihIn\-context reinforcement learning with algorithm distillation\.External Links:2210\.14215Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Leeet al\.\(2020\)J\. Lee, J\. Hwangbo, L\. Wellhausen, V\. Koltun, and M\. HutterLearning quadrupedal locomotion over challenging terrain\.External Links:2010\.11251Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p3.1)\.
- Levyet al\.\(2025\)J\. Levy, J\. Gibson, B\. Vlahov, E\. Tevere, E\. Theodorou, D\. Fridovich\-Keil, and P\. SpielerMeta\-learning online dynamics model adaptation in off\-road autonomous driving\.arXiv preprint arXiv:2504\.16923\.Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1)\.
- Levyet al\.\(2026\)J\. Levy, T\. Westenbroek, K\. Huang, F\. Palafox, P\. Yin, S\. Omidshafiei, D\. Kim, A\. Gupta, and D\. Fridovich\-KeilSimulation distillation: pretraining world models in simulation for rapid real\-world adaptation\.External Links:2603\.15759Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1),[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p1.1)\.
- Liuet al\.\(2025\)M\. Liu, D\. Pathak, and A\. AgarwalLocoFormer: generalist locomotion via long\-context adaptation\.External Links:2509\.23745Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1),[§5\.3](https://arxiv.org/html/2609.12278#S5.SS3.SSS0.Px3.p1.1)\.
- Liuet al\.\(2026\)Y\. Liu, X\. Wang, Y\. Mao, Y\. Gelbery, H\. Maron, and M\. ZhangSHINE: a scalable in\-context hypernetwork for mapping context to lora in a single pass\.External Links:2602\.06358Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Maeset al\.\(2026\)L\. Maes, Q\. Le Lidec, D\. Scieur, Y\. LeCun, and R\. BalestrieroLeWorldModel: stable end\-to\-end joint\-embedding predictive architecture from pixels\.External Links:2603\.19312Cited by:[§3\.1](https://arxiv.org/html/2609.12278#S3.SS1.p3.1)\.
- Mahabadiet al\.\(2021\)R\. K\. Mahabadi, S\. Ruder, M\. Dehghani, and J\. HendersonParameter\-efficient multi\-task fine\-tuning for transformers via shared hypernetworks\.External Links:2106\.04489Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Mishraet al\.\(2017\)N\. Mishra, M\. Rohaninejad, X\. Chen, and P\. AbbeelA simple neural attentive meta\-learner\.External Links:1707\.03141Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Nagabandiet al\.\(2018\)A\. Nagabandi, I\. Clavera, S\. Liu, R\. S\. Fearing, P\. Abbeel, S\. Levine, and C\. FinnLearning to adapt in dynamic, real\-world environments through meta\-reinforcement learning\.arXiv preprint arXiv:1803\.11347\.Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Ortiz\-Barajaset al\.\(2024\)J\. Ortiz\-Barajas, H\. Gomez\-Adorno, and T\. SolorioHyperLoader: integrating hypernetwork\-based lora and adapter layers into multi\-task transformers for sequence labelling\.External Links:2407\.01411Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Penget al\.\(2017\)X\. B\. Peng, M\. Andrychowicz, W\. Zaremba, and P\. AbbeelSim\-to\-real transfer of robotic control with dynamics randomization\.External Links:1710\.06537Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p3.1)\.
- Perezet al\.\(2017\)E\. Perez, F\. Strub, H\. de Vries, V\. Dumoulin, and A\. CourvilleFiLM: visual reasoning with a general conditioning layer\.External Links:1709\.07871Cited by:[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1),[§5\.3](https://arxiv.org/html/2609.12278#S5.SS3.SSS0.Px3.p1.1)\.
- Pfrommeret al\.\(2020\)S\. Pfrommer, M\. Halm, and M\. PosaContactNets: learning discontinuous contact dynamics with smooth, implicit representations\.External Links:2009\.11193Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Przewięźlikowskiet al\.\(2022\)M\. Przewięźlikowski, P\. Przybysz, J\. Tabor, M\. Zięba, and P\. SpurekHyperMAML: few\-shot adaptation of deep models with hypernetworks\.External Links:2205\.15745Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Raghuet al\.\(2019\)A\. Raghu, M\. Raghu, S\. Bengio, and O\. VinyalsRapid learning or feature reuse? towards understanding the effectiveness of maml\.External Links:1909\.09157Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Rakellyet al\.\(2019\)K\. Rakelly, A\. Zhou, D\. Quillen, C\. Finn, and S\. LevineEfficient off\-policy meta\-reinforcement learning via probabilistic context variables\.External Links:1903\.08254Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1),[§5\.3](https://arxiv.org/html/2609.12278#S5.SS3.SSS0.Px3.p1.1)\.
- Reedet al\.\(2022\)S\. Reed, K\. Zolna, E\. Parisotto, S\. G\. Colmenarejo, A\. Novikov, G\. Barth\-Maron, M\. Gimenez, Y\. Sulsky, J\. Kay, J\. T\. Springenberg, T\. Eccles, J\. Bruce, A\. Razavi, A\. Edwards, N\. Heess, Y\. Chen, R\. Hadsell, O\. Vinyals, M\. Bordbar, and N\. de FreitasA generalist agent\.External Links:2205\.06175Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Rezaei\-Shoshtariet al\.\(2022\)S\. Rezaei\-Shoshtari, C\. Morissette, F\. R\. Hogan, G\. Dudek, and D\. MegerHypernetworks for zero\-shot transfer in reinforcement learning\.External Links:2211\.15457Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Smithet al\.\(2022a\)L\. Smith, J\. C\. Kew, X\. B\. Peng, S\. Ha, J\. Tan, and S\. LevineLegged robots that keep on learning: fine\-tuning locomotion policies in the real world\.In2022 International Conference on Robotics and Automation \(ICRA\),pp\. 1593–1599\.Cited by:[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1)\.
- Smithet al\.\(2022b\)L\. Smith, I\. Kostrikov, and S\. LevineA walk in the park: learning to walk in 20 minutes with model\-free reinforcement learning\.External Links:2208\.07860Cited by:[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1)\.
- Tassaet al\.\(2018\)Y\. Tassa, Y\. Doron, A\. Muldal, T\. Erez, Y\. Li, D\. de Las Casas, D\. Budden, A\. Abdolmaleki, J\. Merel, A\. Lefrancq, T\. Lillicrap, and M\. RiedmillerDeepMind control suite\.External Links:1801\.00690Cited by:[Table 1](https://arxiv.org/html/2609.12278#S5.T1)\.
- Tobinet al\.\(2017\)J\. Tobin, R\. Fong, A\. Ray, J\. Schneider, W\. Zaremba, and P\. AbbeelDomain randomization for transferring deep neural networks from simulation to the real world\.In2017 IEEE/RSJ international conference on intelligent robots and systems \(IROS\),pp\. 23–30\.Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p3.1),[§5\.1](https://arxiv.org/html/2609.12278#S5.SS1.SSS0.Px3.p1.1)\.
- Von Oswaldet al\.\(2020\)J\. Von Oswald, C\. Henning, B\. F\. Grewe, and J\. SacramentoContinual learning with hypernetworks\.In8th International Conference on Learning Representations \(ICLR 2020\)\(virtual\),Cited by:[§4\.1](https://arxiv.org/html/2609.12278#S4.SS1.p4.1)\.
- Wanget al\.\(2025\)Y\. Wang, M\. Verghese, and J\. SchneiderLatent policy steering with embodiment\-agnostic pretrained world models\.External Links:2507\.13340Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Williamset al\.\(2015\)G\. Williams, A\. Aldrich, and E\. TheodorouModel predictive path integral control using covariance variable importance sampling\.External Links:1509\.01149Cited by:[Table 3](https://arxiv.org/html/2609.12278#A2.T3.2.15.1.1),[§3](https://arxiv.org/html/2609.12278#S3.p2.1)\.
- Xianet al\.\(2021\)Z\. Xian, S\. Lal, H\. Tung, E\. A\. Platanios, and K\. FragkiadakiHyperDynamics: meta\-learning object and agent dynamics with hypernetworks\.External Links:2103\.09439Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p1.1)\.
- Xuet al\.\(2025\)J\. Xu, E\. Heiden, I\. Akinola, D\. Fox, M\. Macklin, and Y\. NarangNeural robot dynamics\.External Links:2508\.15755Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1)\.
- Ye and Ren \(2021\)Q\. Ye and X\. RenLearning to generate task\-specific adapters from task description\.External Links:2101\.00420Cited by:[§2\.3](https://arxiv.org/html/2609.12278#S2.SS3.p2.1)\.
- Yinet al\.\(2025\)P\. Yin, T\. Westenbroek, S\. Bagaria, K\. Huang, C\. Cheng, A\. Kobolov, and A\. GuptaRapidly adapting policies to the real world via simulation\-guided fine\-tuning\.External Links:2502\.02705Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p1.1),[§2\.2](https://arxiv.org/html/2609.12278#S2.SS2.p1.1)\.
- Yuet al\.\(2019\)T\. Yu, D\. Quillen, Z\. He, R\. Julian, K\. Hausman, C\. Finn, and S\. LevineMeta\-world: a benchmark and evaluation for multi\-task and meta reinforcement learning\.InConference on Robot Learning \(CoRL\),External Links:1910\.10897Cited by:[Table 1](https://arxiv.org/html/2609.12278#S5.T1)\.
- Yuet al\.\(2017\)W\. Yu, J\. Tan, C\. K\. Liu, and G\. TurkPreparing for the unknown: learning a universal policy with online system identification\.External Links:1702\.02453Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Zhanget al\.\(2024\)X\. Zhang, S\. Liu, P\. Huang, W\. J\. Han, Y\. Lyu, M\. Xu, and D\. ZhaoDynamics as prompts: in\-context learning for sim\-to\-real system identifications\.External Links:2410\.20357Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Zintgrafet al\.\(2018\)L\. M\. Zintgraf, K\. Shiarlis, V\. Kurin, K\. Hofmann, and S\. WhitesonFast context adaptation via meta\-learning\.External Links:1810\.03642Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.
- Zintgrafet al\.\(2019\)L\. Zintgraf, K\. Shiarlis, M\. Igl, S\. Schulze, Y\. Gal, K\. Hofmann, and S\. WhitesonVariBAD: a very good method for bayes\-adaptive deep rl via meta\-learning\.External Links:1910\.08348Cited by:[§2\.1](https://arxiv.org/html/2609.12278#S2.SS1.p2.1)\.