Modality-Gated Deep Adapters: Adding a Modality to a Frozen Embedding Model with Exact Preservation

# Adding a Modality to a Frozen Embedding Model with Exact Preservation
Source: [https://arxiv.org/html/2609.26182](https://arxiv.org/html/2609.26182)
Abdul Basit Tonmoy1,2,3Kazi Fardinul Hoque2Md\. Shahrier Islam Arham1,2Arman Luthra1,2 1Eximius Labs2Wabash College3Skop Intelligence Co\. atonmoy27@wabash\.edukfhfar@amazon\.com marham27@wabash\.edualuthra26@wabash\.edu

###### Abstract

Multimodal embedding models are deployed at scale: retrieval indices, benchmark results, and behavioral audits all depend on the base model’s exact outputs\. Extending such a model to a new modality with existing parameter\-efficient methods silently changes those outputs; LoRA\-style adaptation rewrites the text path whether or not the weights are merged, invalidating every stored embedding\. We propose*modality\-gated deep adapters*: bottleneck adapters attached to every decoder layer of a frozen multimodal embedding LLM, grouped into per\-modality packs that execute only while their own modality is being encoded\. The result is a modality added with zero change to existing outputs: inputs no pack claims traverse the base model’s own computation graph, bit\-for\-bit unchanged, and co\-loaded packs compose with an exact\-zero isolation matrix\. Both properties are stated as propositions, hold after arbitrary training rather than only at initialization, require no task labels or routing metadata at inference, and are verified by exact\-equality tests on the released checkpoints\. On one frozen 2B base, the audio pack \(injected as connector tokens\) improves audio\-to\-text R@10 by\+3\.4\+3\.4to\+5\.4\+5\.4points over an identically trained control, positive at every seed and reproduced at eleven times the data; the thermal pack, reusing the base’s own frozen vision path, clears its pre\-registered acceptance gate roughly sevenfold at every seed and lifts thermal\-to\-text R@10 from 0\.224 to 0\.785\. A motivating encoder swap locates the missing capacity: an external audio encoder that outranks Whisper\-family encoders in CLAP\-style comparisons loses by 16 R@10 points inside the frozen LLM, so the capacity belongs in the layers, exactly where the gated adapters place it\. We release the audio model, the thermal pack, and the training, evaluation, and invariance test suites: models at[https://huggingface\.co/EximiusLabs](https://huggingface.co/EximiusLabs), code on GitHub\.

Modality\-Gated Deep Adapters: Adding a Modality to a Frozen Embedding Model with Exact Preservation

![Refer to caption](https://arxiv.org/html/2609.26182v1/x1.png)Figure 1:One frozen decoder layer with a gated adapter\. Left: on an input whose modality no pack claims \(text, image, video\), the hook returns the frozen layer’s output before any adapter arithmetic executes, so the computation graph is the released base model’s own and the output is bit\-for\-bit the base’s \(Proposition[1](https://arxiv.org/html/2609.26182#Thmproposition1)\); ungated methods such as LoRA instead execute on this path and change it\. Right: on an input of the pack’s own modality \(here audio\), the bottleneck adapter is added to the residual stream and trained\. Each added modality carries its own pack and gate, keyed on the input rather than on a task label, and co\-loaded packs are mutually invisible, so packs compose with exact\-zero isolation \(Table[6](https://arxiv.org/html/2609.26182#S5.T6)\)\.## 1Introduction

An embedding model in production is not just weights; it is a contract\. A deployed embedding model’s outputs are indexed at scale, benchmark tables record its published behavior, and downstream systems are tuned to its geometry\. When such a model gains a new modality, the operative question is not only how well the new modality performs but what happens to that contract\. Retraining produces a new model and a full re\-indexing bill\. Parameter\-efficient adaptation appears gentler, but it is not: LoRA updates, merged or unmerged, execute on every input, so the adapted model is a different function on the modalities users already depend on\(Hu et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib13); Yoo et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib38)\)\. The change is silent; nearest\-neighbor structure shifts without any error being raised\.

Freezing the base is the obvious response, and a family of methods freezes most of it\. Yet frozen\-backbone systems in the literature treat freezing as a compute saving, not a behavioral guarantee\. Omni\-Embed\-Audio\(Yoo et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib38)\)keeps its backbone frozen but trains LoRA in the attention of every layer; the adapters fire on the text path as well, and the deployed function on text is far from the base’s \(the untrained backbone retrieves at roughly chance, the adapted one near state of the art, so the text path was necessarily rewritten\)\. Methods that install zero\-initialized new capacity, such as gated cross\-attention in Flamingo\(Alayrac et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib1)\)and block expansion in LLaMA Pro\(Wu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib34)\), guarantee identity only at initialization; the new modules train away from zero on all inputs\. Such methods report drift on the base’s own tasks once the added modules train, so a preservation property that holds only at initialization does not survive continued training\.

This paper presents a mechanism whose preservation guarantee is exact and permanent\.*Modality\-gated deep adapters*add a bottleneck adapter to the residual stream of every decoder layer of a frozen multimodal embedding LLM, and a binary gate, keyed on the presence of audio in the input rather than on any task label, decides whether the adapter branch executes at all \(Figure[1](https://arxiv.org/html/2609.26182#S0.F1)\)\. While audio is encoded the adapters are active and trained; for any input without audio every adapter hook returns the frozen layer’s output before any adapter arithmetic, so the executed computation graph is the base model’s own and the output is bit\-for\-bit identical to the released base \(Proposition[1](https://arxiv.org/html/2609.26182#Thmproposition1)\)\. The guarantee is a property of the computation, not of the training trajectory: it holds after arbitrary training, forever, and it is machine\-verified at three points: exact\-equality unit tests, a parameter\-drift assertion after every training run, and a release smoke test on the packaged public checkpoint\.

Three further results make the mechanism more than a safety feature\. First, a motivating probe \(§[2](https://arxiv.org/html/2609.26182#S2)\) shows that the natural alternative, upgrading the external audio encoder, fails: an encoder that ranks above Whisper\-family encoders in CLAP\-style comparisons loses by 16 R@10 points inside the frozen\-LLM architecture\. The bottleneck is in\-layer capacity, precisely what the gated adapters add\. Second, a controlled experiment \(§[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)\) isolates the mechanism: with data, recipe, and connector held fixed, gated adapters improve audio\-to\-text R@10 at every seed tested \(paired deltas\+3\.4\+3\.4to\+5\.4\+5\.4\), the gain scales with adapter rank, and its point estimate reproduced at eleven times the data\. The released model improves the point estimate in every text\-to\-audio retrieval cell on AudioCaps, Clotho, and VGGSound over its non\-adapter predecessor\. Third, the construction is not an audio\-specific trick \(§[5](https://arxiv.org/html/2609.26182#S5)\): a thermal pack added to the same frozen base through a different injection route, the base’s own vision path, clears its pre\-registered gate at every seed, and co\-loading the audio and thermal packs reproduces the base bitwise on every text, image, and video readout\.

Our contributions:

- •Mechanism\.Modality\-gated deep adapters: post\-hoc extension of a frozen multimodal embedding LLM to new modalities, with per\-modality packs of in\-layer trainable capacity that execute only on their own modality’s inputs \(Figure[1](https://arxiv.org/html/2609.26182#S0.F1), §[3](https://arxiv.org/html/2609.26182#S3)\)\. We add audio and thermal to one frozen base through two different injection routes and show the packs compose \(§[5](https://arxiv.org/html/2609.26182#S5)\)\.
- •Property and verification\.Exact preservation, stated as Propositions[1](https://arxiv.org/html/2609.26182#Thmproposition1)and[2](https://arxiv.org/html/2609.26182#Thmproposition2)\(computation\-graph identity for inputs no pack claims, with bitwise equality as the verified corollary\) and enforced by exact\-equality tests, gradient\-isolation tests, a multi\-pack exclusivity invariant, and a released smoke test \(Tables[4](https://arxiv.org/html/2609.26182#S4.T4)and[6](https://arxiv.org/html/2609.26182#S5.T6)\); the guarantee is input\-conditional and permanent, which is strictly stronger than identity at initialization\(Alayrac et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib1); Wu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib34)\)or task\-labeled routing\(Rusu et al\.,[2016](https://arxiv.org/html/2609.26182#bib.bib26); Zhang et al\.,[2020](https://arxiv.org/html/2609.26182#bib.bib39)\)\(§[3\.3](https://arxiv.org/html/2609.26182#S3.SS3), §[4\.4](https://arxiv.org/html/2609.26182#S4.SS4)\)\.
- •Controlled evidence, two modalities\.A same\-recipe audio experiment \(\+3\.4\+3\.4R@10, rank\-scaled, reproduced at full scale; Table[2](https://arxiv.org/html/2609.26182#S4.T2)\), a caption\-free thermal probe that clears its pre\-registered gate at every seed \(§[5](https://arxiv.org/html/2609.26182#S5)\), and a publishable negative \(a leaderboard\-stronger audio encoder degrades the system by 16 R@10 points; Table[1](https://arxiv.org/html/2609.26182#S2.T1)\), which together locate the frozen\-LLM bottleneck in the layers, not the encoder\.
- •Artifact\.The releasedfusion\-embedding\-2audio model and theemberthermal pack \(each 2–3% trained parameters over a byte\-frozen 2B base\), training and evaluation code, and the invariance and composability test suites \(§[9](https://arxiv.org/html/2609.26182#S9)\)\.

## 2Is the audio encoder the bottleneck?

The architecture we extend routes audio through a frozen encoder tower and a small trained connector into a frozen decoder\-LM embedding model \(details in §[3\.1](https://arxiv.org/html/2609.26182#S3.SS1)\)\. Its audio\-text retrieval trails specialist dual encoders, and the most natural diagnosis is the audio encoder: the tower is Whisper\-derived\(Radford et al\.,[2023](https://arxiv.org/html/2609.26182#bib.bib25); Xu et al\.,[2025](https://arxiv.org/html/2609.26182#bib.bib37)\), and controlled comparisons under shallow projections rank Whisper\-family encoders well below sound\-event encoders for sound retrieval\. GLAP’s encoder study\(Dinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6)\)puts CED\-Base at 58\.6 AudioCaps text\-to\-audio mAP@10, Dasheng at 55\.8, and Whisper\-Base at 46\.5 under one recipe\.

We ran the implied experiment\. Two arms, identical in data \(45K AudioCaps\-only\), steps \(800\), recipe, and connector, differ only in the frozen tower supplying audio frames: the base’s own Qwen2\.5\-Omni tower \(Whisper\-family, co\-trained to feed a Qwen LM\) or Dasheng\-base\(Dinkel et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib7)\), the versatile winner of GLAP’s comparison\. The external ranking inverts inside the splice: the Omni tower reaches audio\-to\-text R@10 0\.631 against 0\.469 for Dasheng, with the same ordering in the reverse direction \(0\.684 vs\. 0\.538 text\-to\-audio; Table[1](https://arxiv.org/html/2609.26182#S2.T1)and Figure[2](https://arxiv.org/html/2609.26182#S2.F2), protocol in Appendix[A](https://arxiv.org/html/2609.26182#A1)\)\. The towers are not parameter\-matched \(86M vs\.∼\{\\sim\}640M\), so the probe does not isolate co\-training from scale; what it establishes is that encoder rankings measured under shallow projections do not transfer to architectures that splice tokens into a frozen LLM\. Two adjacent observations reinforce the conclusion: a linear probe of the Omni tower on ESC\-50\(Piczak,[2015](https://arxiv.org/html/2609.26182#bib.bib24)\)rises monotonically to its last layer \(0\.924, with a learned average over all layers worse at 0\.893; from training\-run logs\), so there is no hidden mid\-stack feature to tap\(Gong et al\.,[2023](https://arxiv.org/html/2609.26182#bib.bib10)\); and the one controlled same\-connector study we know of finds that adapting a Whisper\-family encoder buys nothing for audio captioning \(LoRA 44\.8 vs\. frozen 44\.8 SPIDEr\-FL, full fine\-tuning worse at 44\.0\) while a frozen event encoder reaches 49\.6\(Liu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib19)\)\.

Table 1:An encoder that outranks Whisper\-family towers in shallow\-projection comparisons\(Dinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6)\)loses by 16 points inside the frozen\-LLM splice\. Matched 45K/800\-step arms, identical recipe, each arm on frames from its own frozen tower\. The towers are not parameter\-matched; the probe establishes non\-transfer of the external ranking, not encoder superiority per parameter\.![Refer to caption](https://arxiv.org/html/2609.26182v1/x2.png)Figure 2:The ranking inversion behind Table[1](https://arxiv.org/html/2609.26182#S2.T1)\. Left: under shallow projections, Dasheng outranks Whisper\-family encoders on sound retrieval \(AudioCaps text\-to\-audio mAP@10, as reported byDinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6)\)\. Right: spliced into the frozen LLM under a matched recipe, the base’s co\-trained Whisper\-family tower wins by 16 R@10 points\. External encoder rankings do not transfer to this architecture class\.If the encoder is not the bottleneck, the natural remaining suspect is the frozen LM itself: every layer of it was trained on text and images, and all audio understanding must squeeze through a 16\.4M input\-side connector into layers that never learned to process audio\. Capacity must be added inside the layers, on the audio path\. Adding it the standard way \(LoRA in the attention\) rewrites the model on every input; adding it behind a modality gate does not\. That is the mechanism\.

## 3Modality\-gated deep adapters

This section defines the mechanism and proves its central property, exact preservation \(Proposition[1](https://arxiv.org/html/2609.26182#Thmproposition1)\): an input no pack claims executes the base model’s own computation graph, so its output is bit\-for\-bit the released base’s\.

### 3\.1Setup and notation

The base model is Qwen3\-VL\-Embedding\-2B\(Li et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib18)\), a 28\-layer decoder\-LM embedding model with hidden widthd=2048d=2048, last\-token pooling, and Matryoshka output truncation\(Kusupati et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib16)\); it embeds text, images, and video in one space and is kept byte\-frozen throughout \(a regression guard asserts that no base parameter changed after every run\)\. Audio enters through the frozen Qwen2\.5\-Omni audio tower\(Xu et al\.,[2025](https://arxiv.org/html/2609.26182#bib.bib37)\)and a trained 16\.4M perceiver\-resampler connector that writes 64 audio tokens into the base’s input stream at placeholder positions; this connector\-only configuration is the releasedfusion\-embedding\-1\(Tonmoy et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib29)\)and serves as the architecture our adapters extend\. Training is symmetric InfoNCE\(van den Oord et al\.,[2018](https://arxiv.org/html/2609.26182#bib.bib30)\)over audio\-caption pairs against cached frozen text targets\. Lethℓ∈ℝn×dh\_\{\\ell\}\\in\\mathbb\{R\}^\{n\\times d\}be the hidden states after decoder layerℓ\\ell, and letg​\(x\)∈\{0,1\}g\(x\)\\in\\\{0,1\\\}indicate whether inputxxcontains audio tokens:ggis a fixed function of the input’s composition, not a learned or task\-supplied router\.

### 3\.2The mechanism

At every decoder layerℓ\\ellwe attach a bottleneck adapter\(Houlsby et al\.,[2019](https://arxiv.org/html/2609.26182#bib.bib12)\)

Aℓ​\(h\)=Wℓup​σ​\(Wℓdown​LN​\(h\)\),A\_\{\\ell\}\(h\)=W^\{\\mathrm\{up\}\}\_\{\\ell\}\\,\\sigma\\\!\\bigl\(W^\{\\mathrm\{down\}\}\_\{\\ell\}\\,\\mathrm\{LN\}\(h\)\\bigr\),\(1\)withWℓdown∈ℝr×dW^\{\\mathrm\{down\}\}\_\{\\ell\}\\in\\mathbb\{R\}^\{r\\times d\},Wℓup∈ℝd×rW^\{\\mathrm\{up\}\}\_\{\\ell\}\\in\\mathbb\{R\}^\{d\\times r\},σ\\sigmathe SiLU nonlinearity, andLN\\mathrm\{LN\}a LayerNorm\. The adapter output is added to the residual stream, but only through a gate:

hℓ←\{hℓ\+Aℓ​\(hℓ\)if​g​\(x\)=1,hℓif​g​\(x\)=0\.h\_\{\\ell\}\\leftarrow\\begin\{cases\}h\_\{\\ell\}\+A\_\{\\ell\}\(h\_\{\\ell\}\)&\\text\{if \}g\(x\)=1,\\\\\[2\.0pt\] h\_\{\\ell\}&\\text\{if \}g\(x\)=0\.\\end\{cases\}\(2\)The gate is implemented as a depth\-counted context manager held open by the model wrapper around audio encodes; the adapters are attached with forward hooks and owned by the wrapper, never registered under the base module, so the base’s parameter snapshot andstate\_dictremain adapter\-free\. Figure[1](https://arxiv.org/html/2609.26182#S0.F1)shows the mechanism; the implementation of the gated hook is short enough to give in full \(Figure[3](https://arxiv.org/html/2609.26182#S3.F3)\)\. At rankr=384r=384over 28 layers the adapters total 44\.2M parameters, which together with the 16\.4M connector gives 60\.6M trained parameters, about 3% of the 2B base\.

def\_make\_hook\(adapter,gate\):

defhook\(\_module,\_inputs,output\):

ifnotgate\.active:

returnNone

ifisinstance\(output,tuple\):

h=output\[0\]

return\(\(h\+adapter\(h\),\)

\+tuple\(output\[1:\]\)\)

returnoutput\+adapter\(output\)

returnhook

Figure 3:The gated forward hook from the released code \(type annotations and comments shortened; indentation compressed for the column\)\. When the gate is closed the hook returns before any adapter computation, so the frozen layer’s output is used unchanged; the tuple branch handles decoder layers that return\(hidden, \.\.\.\)\.
### 3\.3Exact preservation

The mechanism generalizes past a single modality\. A*pack*mmis a full per\-layer adapter stack \(Eq\.[1](https://arxiv.org/html/2609.26182#S3.E1)\) with its own binary gategm​\(x\)g\_\{m\}\(x\); several packs may share the same frozen layers, each registering its own gated hook\. The released model carries an audio pack, and §[5](https://arxiv.org/html/2609.26182#S5)adds a thermal pack to the same base\. Letfbasef\_\{\\mathrm\{base\}\}be the released base andffthe model with any set of packs attached\.

###### Proposition 1\(Exact preservation\)\.

For every inputxxwithgm​\(x\)=0g\_\{m\}\(x\)=0for all packsmm,ffexecutes the same operations asfbasef\_\{\\mathrm\{base\}\}, on the same weights, in the same order; hencef​\(x\)=fbase​\(x\)f\(x\)=f\_\{\\mathrm\{base\}\}\(x\)exactly\.

###### Proof\.

The only modifications to the base are the packs’ layer hooks\. With every gate closed, each hook returns the frozen layer’s output before any adapter operation enters the computation \(Eq\.[2](https://arxiv.org/html/2609.26182#S3.E2), Figure[3](https://arxiv.org/html/2609.26182#S3.F3)\); no adapter parameter, activation, or cast of any pack participates\. The executed computation graph is thereforefbasef\_\{\\mathrm\{base\}\}’s own, and the outputs coincide exactly\. ∎

A companion invariant makes the multi\-pack case precise\.

###### Proposition 2\(Single\-gate exclusivity\)\.

Fix an inputxxand suppose exactly one packm⋆m^\{\\star\}hasgm⋆​\(x\)=1g\_\{m^\{\\star\}\}\(x\)=1while every other gate is closed\. Thenf​\(x\)f\(x\)equals the output of the model carrying packm⋆m^\{\\star\}alone: co\-loaded packs are mutually invisible, because each closed pack’s hook is a bitwise no\-op\.

We scope the invariant deliberately to*at most one gate active per forward*\. A pathological input that opened two gates at once would have both packs add to the residual stream, an interaction we neither train nor claim; because each modality is encoded through its own entry point \(below\), this case does not arise in normal use and we exclude it \(§[8](https://arxiv.org/html/2609.26182#S8)\)\. What the invariant covers is the deployment mix: an audio encode fires only the audio pack, a thermal encode only the thermal pack, and any text, image, or video forward fires neither, reproducing the base bitwise \(§[5](https://arxiv.org/html/2609.26182#S5), Table[6](https://arxiv.org/html/2609.26182#S5.T6)\)\.

Two scope statements make the claim precise\. First, bitwise output equality follows under any fixed inference configuration \(same weights, precision, kernels, and batch composition\); across configurations the model is exactly as deterministic as the base, because on inputs no pack claims it*is*the base’s computation\. Second, the property is input\-conditional: a sequence that a pack does claim is adapted as a whole, including its text tokens\. Such inputs are new behavior by definition; the base model has no audio \(or thermal\) interface, so there is no base behavior to regress\. We deliberately state the property as “the branch never executes” rather than “the branch adds zero,” sinceh\+0⋅A​\(h\)h\+0\\cdot A\(h\)would still change the graph and invite floating\-point concerns; a hook that returns early does not\.

Each gate is*declared by the encode entry point*rather than inferred from tensor contents\. This matters most for thermal: a thermal image is a single channel replicated to three, byte\-indistinguishable from an ordinary RGB image at the vision interface, so no content test could route it\. The thermal encode is instead the only code path that opens the thermal gate, and it closes it on exit; every image, video, and text embed runs outside that scope with the gate closed\. For audio the declaration is reinforced by content: audio tokens are present only on the audio path, and the public text\-encode and image\-embed entry points hard\-fail if invoked while the audio gate is open \(in the training and released inference code respectively\)\. The released inference guard covers the audio gate; thermal preservation rests on the encode\-scoping discipline just described, which the isolation matrix of §[5](https://arxiv.org/html/2609.26182#S5)verifies bitwise\. The gatesgmg\_\{m\}in Propositions[1](https://arxiv.org/html/2609.26182#Thmproposition1)and[2](https://arxiv.org/html/2609.26182#Thmproposition2)are thus properties the encode entry points guarantee, not heuristics over inputs\.

Proposition[1](https://arxiv.org/html/2609.26182#Thmproposition1)is stronger than the preservation notions in prior expansion methods on two independent axes\. On the*time*axis, zero\-initialized capacity\(Alayrac et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib1); Bachlechner et al\.,[2021](https://arxiv.org/html/2609.26182#bib.bib2); Wu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib34)\)matches the base only at initialization; our guarantee is invariant to training, because training only ever touches the gated branch\. On the*routing*axis, progressive networks\(Rusu et al\.,[2016](https://arxiv.org/html/2609.26182#bib.bib26)\)and side\-tuning\(Zhang et al\.,[2020](https://arxiv.org/html/2609.26182#bib.bib39)\)preserve frozen components but select the active path with task labels supplied at inference; our gate is a function of the input alone\. Unmerged LoRA preserves base*parameters*, but its adapters execute on every path, so the deployed function changes on all inputs\(Hu et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib13); Yoo et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib38)\); parameter preservation without computation preservation is the weaker guarantee, and the distinction is exactly what re\-indexing costs measure\. Section[4\.4](https://arxiv.org/html/2609.26182#S4.SS4)verifies the property empirically, including the negative control \(with the gate open, outputs change; that visible change is what ungated adaptation applies to every input silently\)\.

### 3\.4Identity at initialization

To ensure that at initialization the adapted model produces exactly the outputs of the base model on*all*inputs, including audio, the up\-projectionsWℓupW^\{\\mathrm\{up\}\}\_\{\\ell\}are zero\-initialized, following the near\-identity principle ofHoulsby et al\. \([2019](https://arxiv.org/html/2609.26182#bib.bib12)\)and the zero\-init gating ofAlayrac et al\. \([2022](https://arxiv.org/html/2609.26182#bib.bib1)\)andBachlechner et al\. \([2021](https://arxiv.org/html/2609.26182#bib.bib2)\): at step zeroAℓ​\(h\)=0A\_\{\\ell\}\(h\)=0and the audio path behaves as the connector\-only architecture, so training starts from the predecessor model rather than from a perturbed one, which stabilizes optimization and makes warm\-starting from a connector\-only checkpoint well\-defined\. The two properties are complementary and should not be conflated: identity at initialization holds for all inputs and is temporary; exact preservation holds for inputs that no pack claims \(audio\-free and thermal\-free\) and is permanent\.

### 3\.5The gate must span forward and backward

One implementation subtlety carries the guarantee in practice\. With gradient checkpointing, layer forwards re\-run during the backward pass\. A gate that is open for the forward pass but closed by backward time silently drops the adapters from the recomputed graph: gradients are wrong and no error is raised, unless the checkpointing implementation detects the mismatch\. We therefore hold the gate open across forward and backward of every audio step \(the depth\-counted design makes the trainer’s scope and the encoder’s inner scope compose\), rely on non\-reentrant checkpointing to raise loudly on any mismatch, and lock the failure mode in with a dedicated test\. The subtlety is also evidence that the property has content: naive gating does not provide the guarantee, and a correct implementation must be verified, which is why the test suite of §[4\.4](https://arxiv.org/html/2609.26182#S4.SS4)exists\. A second guard closes the loop from the other side: a text encode issued while the gate is open raises immediately, so the frozen text targets used by the contrastive objective can never be silently adapted\.

### 3\.6Design choices

Adapters attach at every layer because the probe of §[2](https://arxiv.org/html/2609.26182#S2)locates the deficit in the frozen stack generally, and because layer subsets were not needed to pass the pre\-registered gate; layer\-placement ablations remain open \(§[8](https://arxiv.org/html/2609.26182#S8)\)\. The rank is chosen empirically \(r=384r=384; §[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)sweeps it\)\. The adapter computes in fp32 and casts back to the stream’s dtype for stability at bf16 scale\. The gate keys on audio because audio is the modality being added; nothing in the construction is audio\-specific, and the same gating applies to any modality appended to a frozen base\.

## 4Experiments

### 4\.1Setup

#### Training\.

All runs pair the frozen base and tower with the 16\.4M connector; adapter runs add gated adapters per §[3](https://arxiv.org/html/2609.26182#S3)\. The controlled experiment trains on a 45K AudioCaps\-only corpus\(Kim et al\.,[2019](https://arxiv.org/html/2609.26182#bib.bib15)\)for 800 steps; full\-scale runs train on 518,183 audio\-caption pairs \(a 592K web corpus after removing 73,716 clips whose metadata carried no sound content\) for 3,900 steps at effective batch 1,024, with a full\-corpus frozen\-text negative bank and large\-scale loss terms\. What that corpus does and does not teach the audio path, and why its structure rather than its size governs which attributes survive, is analyzed separately\(Tonmoy,[2026](https://arxiv.org/html/2609.26182#bib.bib28)\)\. A 400\-step in\-domain AudioCaps fine\-tune follows, as in the predecessor\. Precision lineage is matched end to end \(train, fine\-tune, and score at one precision\); mixing lineages costs about 2 points and is a documented trap\. Full hyperparameters are in Appendix[D](https://arxiv.org/html/2609.26182#A4)\.

#### Evaluation\.

Two protocols are used, never mixed\. The*in\-run protocol*is the automated AudioCaps rescore every training job runs at its own precision; it is used for matched\-arm comparisons\. The*release protocol*\(bf16, the base’s native input template, five\-reference min\-rank scoring on AudioCaps\-883; Clotho v2\.1\(Drossos et al\.,[2020](https://arxiv.org/html/2609.26182#bib.bib8)\)strictly zero\-shot; VGGSound\-696\(Chen et al\.,[2020](https://arxiv.org/html/2609.26182#bib.bib4)\)for cross\-modal cells\) is used for released checkpoints\. Clotho never appears in training; VGGSound is blacklisted from ingestion, and 5,929 Clotho\-overlapping FreeSound clip ids are excluded from the web corpus\. At these pool sizes a recall proportion carries a binomial standard error of roughly 1\.4–1\.5 points, so we read single deltas below∼2\{\\sim\}2points as parity\. The controlled gate experiment is replicated across three seeds \(§[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)\); the full\-scale pretrain pair and the release checkpoints are single runs\.

### 4\.2The controlled gate experiment

The causal question is whether gated in\-layer capacity improves audio retrieval when everything else is held fixed\. Three arms share data, steps, recipe, connector, and tower, and differ in one axis \(Table[2](https://arxiv.org/html/2609.26182#S4.T2)\): no adapters, gated adapters at rank 128, gated adapters at rank 384\. The acceptance gate was pre\-registered at≥\+3\\geq\{\+\}3audio\-to\-text R@10\.

Table 2:Gated in\-layer capacity passes the pre\-registered acceptance gate \(≥\+3\\geq\{\+\}3R@10\) at\+3\.4\+3\.4, improves every measured direction, and has not saturated at the largest rank trained\. The controlled experiment: 45K AudioCaps\-only, 800 steps, identical recipe, one change per row; in\-run protocol; audio\-to\-text unless marked; final training loss in the last column\. Rank 384 is the released configuration\. Dashes: not recorded\.Gated adapters at rank 384 improve audio\-to\-text R@10 by\+3\.4\+3\.4points and every other measured cell, with a lower final training loss; rank 128 recovers\+2\.5\+2\.5, so the gain scales with adapter capacity and had not saturated\. Both deltas exceed the∼2\{\\sim\}2\-point single\-seed parity threshold of §[4\.1](https://arxiv.org/html/2609.26182#S4.SS1)\. The comparison isolates the adapters as the only changed component; it does not equalize trained\-parameter count across arms\. Two observations address the budget question directly\. First, spending a comparable increment at the input instead does not help: in the predecessor’s width study, growing the connector fromdr=384d\_\{r\}\{=\}384todr=512d\_\{r\}\{=\}512matched or*reduced*held\-out retrieval at the two scales tested \(0\.479 vs\. 0\.481 at 131K; 0\.675 vs\. 0\.717 R@10 at 484K\) despite better training metrics\. Second, that width increase and the rank\-128 arm are similarly sized parameter increments \(11\.8M and 14\.8M\); spent in\-layer behind the gate the increment gains\+2\.5\+2\.5, spent on connector width it gains nothing\. Where the parameters live matters more than how many there are: input\-side width was already past its knee, while in\-layer capacity was absent entirely\.

#### Seed replication\.

Re\-running both arms of Table[2](https://arxiv.org/html/2609.26182#S4.T2)with a seeded variant of the identical recipe \(the seed enters the training run’s resume key; the original arms predate the seed parameter\) replicates the effect\. Across three seeds the paired audio\-to\-text R@10 delta is\+3\.4\+3\.4/\+4\.6\+4\.6/\+5\.4\+5\.4\(mean\+4\.5\+4\.5; control 0\.631/0\.608/0\.595, adapters 0\.665/0\.655/0\.649\); text\-to\-audio R@10 deltas are\+2\.4\+2\.4/\+4\.6\+4\.6/\+5\.2\+5\.2and audio\-to\-text R@1 deltas\+2\.7\+2\.7/\+3\.9\+3\.9/\+5\.1\+5\.1\. Every retrieval direction is positive at every seed, and the pairing published in Table[2](https://arxiv.org/html/2609.26182#S4.T2)is the most conservative of the three\.

At eleven times the data the point estimate reproduces\. With the 518,183\-pair filtered corpus and large\-scale loss terms, the adapter pretrain reaches in\-run R@10 0\.708 against 0\.674 for the closest available baseline, the 592K raw\-corpus run without adapters or the loss terms \(matched evaluation protocol\):\+3\.4\+3\.4again\. The full\-scale pair differs on three axes \(adapters, loss terms, and corpus filtering\); the controlled isolation is Table[2](https://arxiv.org/html/2609.26182#S4.T2), and the full\-scale pair shows the gain’s magnitude survives scale and recipe maturation\.

### 4\.3Release results: the family delta

The released adapter model improves the point estimate in every text\-to\-audio cell over its connector\-only predecessor under the release protocol \(Table[3](https://arxiv.org/html/2609.26182#S4.T3)\)\. The pattern is directional and consistent with the mechanism: in\-layer capacity lets the audio representation organize for discrimination against text queries \(R@10: AudioCaps\+2\.9\+2\.9, Clotho\+2\.2\+2\.2, VGGSound\+3\.6\+3\.6; VGGSound R@1\+5\.3\+5\.3; the AudioCaps R@1 gain of\+1\.2\+1\.2and the Clotho R@1 gain of\+1\.5\+1\.5are below the parity threshold\), at a small cost in audio\-to\-text top\-1 on the caption benchmarks\. Audio\-to\-text R@10 on AudioCaps is statistical parity \(0\.743 vs\. 0\.741\)\. Against external systems the released model remains behind specialist dual encoders on in\-domain audio\-text retrieval \(GLAP reports 0\.544/0\.911 AudioCaps audio\-to\-text R@1/R@10\(Dinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6)\)\); those systems train both towers on audio\-text data and serve exactly one modality pair, and no result in this paper claims otherwise\. What the adapter model uniquely retains is the rest of the space: its text, image, and video behavior is the base model’s, bitwise, which no specialist or LoRA\-adapted system provides\.

CellPredecessorOurs*AudioCaps*A→\\toT R@1 / R@100\.332/ 0\.741†0\.302 /0\.743†T→\\toA R@1 / R@100\.280†/ 0\.7460\.292†/0\.775*Clotho \(zero\-shot\)*A→\\toT R@1 / R@100\.135†/0\.433†0\.127†/ 0\.421†T→\\toA R@1 / R@100\.136†/ 0\.4600\.151†/0\.482*VGGSound\-696*A→\\toT R@1 / R@100\.213†/ 0\.6250\.211†/0\.665T→\\toA R@1 / R@100\.213 / 0\.6450\.266/0\.681Table 3:The adapters improve the point estimate in every text\-to\-audio cell and the VGGSound audio\-to\-text R@10, and cede audio\-to\-text R@1 on the caption benchmarks\. Release\-protocol comparison against the connector\-only predecessor \(fusion\-embedding\-1v0\.3\), matched bf16 lineage and native input template; Clotho strictly zero\-shot\. Bold: better point estimate per cell;†marks both members of a pair whose difference is below the∼2\{\\sim\}2\-point single\-seed parity threshold \(§[4\.1](https://arxiv.org/html/2609.26182#S4.SS1)\), read as parity\.The emergent cross\-modal direction is also present\. With zero audio\-image training pairs, audio\-to\-image R@10 on VGGSound reaches 0\.443 for the adapter pretrain \(bare input template\) and 0\.392 for its native\-template fine\-tuned successor, both far above chance\. We do not read a gain from the predecessor’s native\-template release into this, since template differences alone shift retrieval by about two points; the point is only that the adapters retain the emergent audio\-image bridge that the audio\-to\-text alignment rides on\.

### 4\.4Preservation verification

Every check that must return zero returns exactly zero, and every control that must differ does \(Table[4](https://arxiv.org/html/2609.26182#S4.T4)\); all checks are released with the code\.

Table 4:Verification of Proposition[1](https://arxiv.org/html/2609.26182#Thmproposition1)and its supporting machinery\. Zeros are exact \(maximum absolute difference under matched execution\), not tolerances\. The open\-gate row is the negative control: outputs must change when the branch executes, and that visible, gated change is what ungated adaptation applies to every input\. Artifact\-level detail in Appendix[B](https://arxiv.org/html/2609.26182#A2)\.The exact\-equality unit tests encode identical text through the base and through the adapter\-attached model and assert bitwise\-equal activations \(maximum absolute difference zero, not small\)\. The release smoke test embeds identical text and image inputs with the adapter hooks attached \(gate closed\) and with the hooks removed, through the public loading path, and requires bitwise\-equal outputs on both modalities\. The regression guard’s drift assertion returned zero on every training run reported here\. The controls that must differ, do: an open gate changes audio\-gated forwards, and only adapter and connector parameters receive gradients\. A downstream consequence makes the guarantee visible in retrieval tables: text\-image retrieval cells for the predecessor and the adapter model are identical because the underlying vectors are identical to the last bit, and the base’s published text/image/video benchmark results\(Li et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib18); Meng et al\.,[2025](https://arxiv.org/html/2609.26182#bib.bib21)\)remain true of the extended model by construction rather than by re\-measurement\.

## 5A second modality: thermal

The construction claims to be modality\-agnostic\. We test that claim by adding a second, deliberately dissimilar modality to the same frozen base through a*different injection route*, and by verifying that the two packs coexist without disturbing each other or the base\.

#### A second injection route\.

Audio enters as connector tokens: an external tower and a trained resampler write new tokens into the input stream\. Thermal imagery enters through the base’s*own*frozen vision path\. A single\-channel thermal frame is replicated to three channels and processed by the unchanged Qwen3\-VL vision encoder and projector; the capacity that the frozen decoder lacks for thermal content is supplied by a thermal pack, a per\-layerGatedAdapterstack \(Eq\.[1](https://arxiv.org/html/2609.26182#S3.E1)\) registered through the multi\-pack registry with its own gate\. No new tower, no new tokens\. The two routes exercise the mechanism differently: the audio gate can key on the presence of audio tokens, but a replicated thermal image is byte\-identical to an RGB image at the vision interface, so the thermal gate*must*be declared by the encode entry point \(§[3\.3](https://arxiv.org/html/2609.26182#S3.SS3)\)\. Thermal is therefore not “the same trick twice”; it stresses the part of the design that does not depend on the new modality being content\-detectable\.

#### Controlled gate probe\.

Mirroring the audio experiment \(§[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)\), a caption\-free probe isolates the thermal pack: two arms identical in data, steps, and recipe \(LLVIP\(Jia et al\.,[2021](https://arxiv.org/html/2609.26182#bib.bib14)\)thermal\-to\-visible\-twin retrieval, rank 384, 800 steps\) differ only in whether the gated thermal pack is attached\. The acceptance gate was pre\-registered at≥\+3\\geq\{\+\}3R@10\. The pack lifts thermal\-to\-image R@10 from a frozen0\.1650\.165to0\.3850\.385/0\.3880\.388/0\.3900\.390across three seeds, a pairedΔ\\Deltaof\+22\.0\+22\.0to\+22\.5\+22\.5points, clearing the pre\-registered gate by roughly sevenfold at every seed; the zero\-shot LLVIP person classifier is unchanged \(95\.4,Δ=0\\Delta=0every seed\), confirming the pack adds capacity without disturbing the frozen vision path it rides on\.

#### Released thermal pack \(Ember\)\.

The deployed pack is trained on IR\-TD, a corpus of 61,320 real thermal images with descriptive captions \(FLIR\-resolution frames stripped for release hygiene\), thermal\-to\-text InfoNCE against cached frozen text targets, rank 384, 3,900 steps, three seeds \(Table[5](https://arxiv.org/html/2609.26182#S5.T5)\)\. Thermal\-to\-text R@10 on a held\-out split rises from0\.2240\.224\(frozen base\) to0\.7770\.777–0\.7850\.785; the emergent thermal\-to\-visible direction, trained on no thermal\-RGB pairs, more than doubles over the frozen base \(0\.165→0\.3330\.165\\to 0\.333–0\.3490\.349\)\. With the thermal pack active, the base’s zero\-shot LLVIP classifier is not degraded \(95\.4 vs\. 94\.1/94\.3/91\.8 across seeds, a one to four point spread\); this is a robustness observation on the gate\-open thermal readout, not the preservation guarantee\. Exact preservation is the separate bitwise check: with the pack gates closed, text and RGB forwards are bitwise\-identical to the base \(Table[6](https://arxiv.org/html/2609.26182#S5.T6)\)\. A caption ablation matches the recaptioning lesson of the family report: at 5,000 steps, full descriptive captions reach in\-domain R@100\.8430\.843and preserve cross\-domain transfer \(LLVIP twin0\.3430\.343\), whereas short class\-name captions reach only0\.5950\.595in\-domain and collapse transfer to0\.0890\.089, below the frozen baseline\. Caption richness matters on both axes\.

Table 5:The released thermal pack \(Ember\) lifts thermal\-to\-text R@10 from 0\.224 to 0\.777–0\.785 and more than doubles the emergent thermal\-to\-visible direction, at every seed\. Three seeds versus the frozen base; thermal\-to\-text is the IR\-TD held\-out split; the frozen column is the same base with the pack’s zero\-initialized up\-projection \(exact identity\)\.∗Thermal\-to\-visible \(LLVIP twin\) is emergent: no thermal\-RGB pairs appear in training\. The zero\-shot classifier row shows the pack does not degrade the base’s thermal zero\-shot classification \(a gate\-open readout\); exact preservation of non\-thermal inputs is the bitwise check in Table[6](https://arxiv.org/html/2609.26182#S5.T6)\.
#### Composability: an exact isolation matrix\.

The two packs are co\-loaded on the same frozen decoder layers exactly as they would be served together: the released audio pack through the shipped single\-gate attachment, the released thermal pack through the multi\-pack registry\. We then embed one input per modality and compare, bitwise, against the reference that Proposition[1](https://arxiv.org/html/2609.26182#Thmproposition1)or[2](https://arxiv.org/html/2609.26182#Thmproposition2)predicts, reporting the maximum absolute difference \(Table[6](https://arxiv.org/html/2609.26182#S5.T6)\)\. Every preserved cell is exactly zero, including the video readout, which no prior thermal check had covered; the audio pack driven through the registry code path also matches the shipped path to the last bit\. As a negative control the thermal readout with its gate open differs from the frozen base by0\.100\.10, so the zeros are not a dead pack\. The matrix is the machine\-checked form of the deployment claim: adding thermal to the audio model changes nothing a user already depends on\.

ReadoutGate\(s\) openmax\|Δ\|\|\\Delta\|Textnone0Image \(RGB\)none0Videonone0Audioaudio0Thermalthermal0Audio, registry pathaudio0Thermal \(control\)thermal, vs\. base0\.100\.10Table 6:Every preserved readout is exactly zero: the released audio and thermal packs, co\-loaded on one frozen base, reproduce their reference bit\-for\-bit\. Each readout is compared to the reference the propositions predict \(text/image/video to the raw base, audio to the audio\-pack\-only model, thermal to the thermal\-pack\-only model\); max\|Δ\|\|\\Delta\|is the exact maximum absolute difference,0meaning bitwise\-identical\. The last two rows are checks: the audio pack through the registry vs\. the shipped single\-gate path \(must be0\), and the thermal readout with its gate open vs\. the frozen base \(must be nonzero, confirming the pack is active\)\. Mixed inputs that would open two gates in one forward are outside the guarantee and untested \(§[8](https://arxiv.org/html/2609.26182#S8)\)\.

## 6Analysis

### 6\.1Why does a stronger encoder hurt?

The probe of §[2](https://arxiv.org/html/2609.26182#S2)is not a claim that Dasheng is a weak encoder; on its own benchmarks it is strong\. The frozen LLM, however, is not a generic readout: it consumes token streams shaped like the distribution its co\-training established, and the Omni tower was trained jointly with a Qwen LM to produce exactly such streams\. A stronger but foreign encoder presents features the frozen layers never learned to parse, and no capacity exists downstream to compensate \(that capacity is what §[3](https://arxiv.org/html/2609.26182#S3)adds\)\. The reading is consistent with GLAP’s own finding that encoder rankings are recipe\-relative\(Dinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6)\), and with the captioning result that adapting the encoder does not close the gap\(Liu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib19)\)\. Compatibility with the consumer dominates quality of the producer\.

### 6\.2What does the gate cost?

On non\-audio inputs, nothing: no added operations, no added latency, no added memory beyond holding the \(inactive\) adapter weights, and exact output stability, which is the point\. On audio inputs the adapters add one bottleneck per layer \(60\.6M trained parameters in total, about 3% of the 2B base\); audio encoding was already the expensive path through a 640M tower, and the adapter increment is minor by comparison\. At training time the practical cost is the discipline of §[3\.5](https://arxiv.org/html/2609.26182#S3.SS5): the gate must be scoped across forward and backward, and the text\-encode guard forces cached text targets\. We consider both constraints features; each converts a silent failure mode into a loud one\.

### 6\.3Relation to continual learning

On its original modalities the extended model does not approximately retain the base’s behavior; it reproduces it exactly, because those inputs execute the base’s computation\. In continual\-learning terms, the mechanism has exact recall of base\-model outputs on audio\-free inputs by construction, echoing the frozen\-parameter memory arguments ofHoulsby et al\. \([2019](https://arxiv.org/html/2609.26182#bib.bib12)\)andRusu et al\. \([2016](https://arxiv.org/html/2609.26182#bib.bib26)\), while avoiding both the task\-labeled routing those methods require and the after\-training drift of zero\-init expansion\(Wu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib34)\)\. We scope the claim deliberately: it concerns embedding outputs on audio\-free inputs, not any broader notion of capability retention, and mixed audio\-text inputs are new behavior with no base counterpart\. Preservation language should be scoped precisely rather than asserted broadly, and ours is checkable by running the released tests\.

## 7Related work

#### Parameter\-efficient adaptation\.

Bottleneck adapters\(Houlsby et al\.,[2019](https://arxiv.org/html/2609.26182#bib.bib12)\)and LoRA\(Hu et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib13)\)add small trained deltas to frozen models; both execute on every input, so the adapted model is a different function everywhere\. Our adapter is Houlsby’s module behind a modality gate; the gate, not the bottleneck, is what produces the guarantee\.

#### Gated injection into frozen LMs\.

Flamingo interleaves tanh\-gated cross\-attention into a frozen LM withα=0\\alpha=0at initialization\(Alayrac et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib1)\), following zero\-init residual principles\(Bachlechner et al\.,[2021](https://arxiv.org/html/2609.26182#bib.bib2)\); the gates train away from zero for all inputs\. Frozen\-tower bridges such as BLIP\-2 train a querying transformer between frozen unimodal models\(Li et al\.,[2023](https://arxiv.org/html/2609.26182#bib.bib17)\); the deployed embedding function is the bridge’s, not the base’s, for every input\. Our gate is binary and input\-keyed rather than learned, which is why it can be exact\.

#### Frozen\-backbone multimodal embedders\.

Closest in setting is jina\-embeddings\-v5\-omni\(Hönicke et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib11)\), which keeps a text backbone frozen and trains only projector\-style connectors \(about 0\.35% of weights\) to bind image, audio, and video into the text space\. This is preservation*by omission*: nothing is added inside the frozen layers, so the original space is untouched, but the new modalities are limited to what shallow projections can align\. Our packs preserve the base equally \(the closed\-gate path*is*the frozen computation\) while adding trainable capacity*inside*the layers on the new modality’s path, and the audio probe of §[2](https://arxiv.org/html/2609.26182#S2)is direct evidence that in\-layer capacity, not a better projector, is what the frozen\-LLM setting is missing\. X\-InstructBLIP\(Panagopoulou et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib23)\)aligns several modalities \(image, 3D, audio, video\) to a frozen LLM through per\-modality Q\-Formers on the input side; preservation there is trivial, in that the backbone is never touched and no exactness property for an unmodified path is stated or tested, and, as with all input\-side composition, the new modalities’ quality is bounded by what tokens a frozen stack can consume\. Our contribution is complementary: an in\-layer, gated capacity whose bypass carries a machine\-checked exactness guarantee and whose packs compose \(§[5](https://arxiv.org/html/2609.26182#S5)\)\. Both routes coexist in the released family this work extends: inertial motion \(Tremor\) and tactile pressure \(Tactus\) bind to the same frozen space through external encoder\-plus\-projector heads, since a small external encoder suffices for those signals, while audio and thermal take the in\-layer route because their inputs must traverse the frozen stack itself\.111All packs are released at[https://huggingface\.co/EximiusLabs](https://huggingface.co/EximiusLabs)\.Which route a modality needs is decided by its path through the base; the packs here supply the case the projector route cannot\.

#### Model expansion and routing\.

LLaMA Pro’s block expansion is identity at initialization and drifts with training\(Wu et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib34)\); Net2Net’s function\-preserving transforms are deliberately a warm start\(Chen et al\.,[2016](https://arxiv.org/html/2609.26182#bib.bib5)\); progressive networks\(Rusu et al\.,[2016](https://arxiv.org/html/2609.26182#bib.bib26)\)and side\-tuning\(Zhang et al\.,[2020](https://arxiv.org/html/2609.26182#bib.bib39)\)preserve frozen components exactly but select paths with task labels at inference\. Multi\-adapter serving systems such as S\-LoRA\(Sheng et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib27)\)also execute adapters conditionally, hot\-swapping them per request; there the routing signal is deployment metadata attached to the request, whereas our gate is a function of the input’s own token composition, and none of these systems states or verifies an exactness property for the unadapted path\. We obtain single\-model deployment with a guarantee that is exact, input\-conditional, and permanent\. LLaMA Pro’s stated open problem, extending to modalities while maintaining original ability, is the setting solved here for embedding models\.

#### Routing and modality experts\.

Mixture\-of\-modality\-experts architectures such as VLMo\(Bao et al\.,[2022](https://arxiv.org/html/2609.26182#bib.bib3)\)route tokens to modality\-specific expert FFNs, and mixture\-of\-experts routing generally allocates per\-token capacity; in all of these the experts are trained jointly with the backbone from the start\. Our adapters are post\-hoc additions to a frozen, already\-deployed base, and the bypass path carries an exactness guarantee rather than a learned routing decision\.

#### Audio\-text embedding models\.

Specialist dual encoders set the pace on audio\-text retrieval\(Wu et al\.,[2023](https://arxiv.org/html/2609.26182#bib.bib35); Mei et al\.,[2024](https://arxiv.org/html/2609.26182#bib.bib20); Dinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6); Niizumi et al\.,[2025](https://arxiv.org/html/2609.26182#bib.bib22)\), and audio\-native LLM retrievers push it further with in\-backbone LoRA and re\-ranking\(Yoo et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib38); Xu et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib36)\); none serves modalities beyond audio and text, and none preserves a base model’s behavior\. Space\-stitching methods\(Wang et al\.,[2025](https://arxiv.org/html/2609.26182#bib.bib33),[2024](https://arxiv.org/html/2609.26182#bib.bib32)\)combine pre\-trained spaces with trained projectors, changing every constituent’s outputs\. ImageBind\(Girdhar et al\.,[2023](https://arxiv.org/html/2609.26182#bib.bib9)\)and ONE\-PEACE\(Wang et al\.,[2023](https://arxiv.org/html/2609.26182#bib.bib31)\)train unified spaces from scratch rather than extending a deployed one\.

## 8Limitations

#### One base family, two modalities\.

Evidence comes from two modalities, audio and thermal, added to one 2B base through two different injection routes \(connector tokens and the frozen vision path\), and the two packs compose bitwise \(§[5](https://arxiv.org/html/2609.26182#S5)\)\. What remains untested is generality across base*families*: every result uses the same frozen Qwen3\-VL backbone, so we do not claim the mechanism transfers to a different embedding LLM without re\-verification\. Scaling past two co\-loaded packs is likewise unmeasured\.

#### Audio ceiling\.

Gains are bounded by what a frozen 2B LM can do with audio tokens: specialist systems that train both towers remain well ahead on in\-domain audio\-text R@1 \(§[4\.3](https://arxiv.org/html/2609.26182#S4.SS3)\)\.

#### Scope of the bitwise claim\.

Bitwise equality is asserted under matched inference configuration; across kernels or hardware the model inherits exactly the base’s determinism, no more\. Functional identity of the computation graph is unconditional\.

#### Missing ablations\.

Layer\-subset placement, random\-init adapters, and a trained ungated\-adapter arm were not run; the rank sweep and fine\-tune\-length checks are the ablations we have \(Appendix[C](https://arxiv.org/html/2609.26182#A3)\)\. The controlled gate experiment is replicated across three seeds with the effect positive in every direction at every seed \(§[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)\); the full\-scale pretrain pair and the release checkpoints remain single runs, read with the standard\-error convention of §[4\.1](https://arxiv.org/html/2609.26182#S4.SS1)\.

#### Claimed\-modality inputs are new behavior\.

The guarantee covers inputs no pack claims; a sequence a pack does claim \(audio, or a thermal image\), including its text tokens, is adapted and has no base counterpart\. Inputs that would open two gates in one forward are outside Proposition[2](https://arxiv.org/html/2609.26182#Thmproposition2)and untested\.

## 9Conclusion

Modality\-gated deep adapters add a modality to a frozen embedding model while keeping the model, on every input it previously served, exactly itself\. The guarantee is architectural rather than empirical, survives arbitrary training, needs no routing metadata, and is verified by exact\-equality tests shipped with the code; the same experiments that establish it also show the mechanism is where frozen\-LLM audio capacity should go, improving the point estimate in every text\-to\-audio cell over a connector\-only predecessor after a motivating probe ruled out the encoder\. Adding thermal imagery through a second, dissimilar injection route, and showing the two packs compose to an exact\-zero isolation matrix, is evidence that the construction is genuinely modality\-agnostic rather than an audio\-specific trick\. The released artifacts are thefusion\-embedding\-2audio checkpoint \(pinned revisionv0\.1\-preview\) at[https://huggingface\.co/EximiusLabs/fusion\-embedding\-2\-2b\-preview](https://huggingface.co/EximiusLabs/fusion-embedding-2-2b-preview), theemberthermal pack at[https://huggingface\.co/EximiusLabs/fusion\-embedding\-2\-ember](https://huggingface.co/EximiusLabs/fusion-embedding-2-ember), with Apache\-2\.0 training and evaluation code, including the invariance and composability test suites, at[https://github\.com/Eximius\-Labs/fusion\-embedding](https://github.com/Eximius-Labs/fusion-embedding)\.

## Appendix AProbe and controlled\-experiment protocol

#### Tower swap \(§[2](https://arxiv.org/html/2609.26182#S2)\)\.

Both arms: 45K AudioCaps\-train\-only corpus, 800 steps, identical connector architecture, loss, optimizer, schedule, and batch size; each arm ingests and evaluates on frames from its own frozen tower \(Qwen2\.5\-Omni audio tower vs\. Dasheng\-base\)\. Audio\-to\-text cells are from the standard in\-run rescore; text\-to\-audio cells are from the run logs\. Hyperparameters were tuned on the Omni arm and reused unchanged on the Dasheng arm\. GLAP’s encoder numbers quoted for context \(AudioCaps text\-to\-audio mAP@10: CED\-Base 58\.6, BEATs 55\.1, Dasheng 55\.8, Whisper\-Base 46\.5\) are from their published study\(Dinkel et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib6)\)\.

#### Gate experiment \(§[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)\)\.

Same corpus, steps, and recipe as the tower\-swap Qwen\-tower arm, which doubles as the control row of Table[2](https://arxiv.org/html/2609.26182#S4.T2)\. Adapter arms differ from the control only in attaching gated adapters at the stated rank\. The≥\+3\\geq\{\+\}3R@10 acceptance threshold was fixed in the program’s plan document before the runs\. In\-run protocol: the automated AudioCaps rescore at the run’s own precision; arms are compared only within matched configuration\. The seed replication of §[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)reran the control and rank\-384 arms at two further seeds each \(seeding torch, numpy, and the loader shuffle; the seed is part of the run’s resume key\); the full per\-arm grids are in the released result recordadapter\_probe\_multiseed\.json\.

#### Full\-scale reproduction\.

518,183\-pair corpus \(592K minus 73,716 junk\-metadata clips\), soft\-label and false\-negative\-masking loss terms, 3,900 steps, effective batch 1,024, full\-corpus frozen text bank; the baseline is the closest available one, the 592K raw\-corpus run without adapters or the loss terms \(in\-run R@10 0\.674, matched evaluation protocol\), so the pair differs on the three axes noted in §[4\.2](https://arxiv.org/html/2609.26182#S4.SS2)\. The release lineage is bf16 end to end; a 4\-bit lineage of the same recipe measures about 2 points lower when scored at bf16, which is why lineages are never mixed\.

## Appendix BVerification artifacts

The gate isAdapterGateinfusion\_embedding/adapters\.py: a depth\-counted context manager whose hooks return the frozen layer’s output before any adapter arithmetic when closed \(Figure[3](https://arxiv.org/html/2609.26182#S3.F3)\)\. Adapters are owned by the model wrapper, so the base’s parameter snapshot andstate\_dictnever contain them\.tests/test\_adapters\.pyasserts: \(i\) bitwise text invariance through base and adapter\-attached model with the gate closed, and exact identity of a zero\-initialized stack under an open gate; \(ii\) audio\-gated forwards change while a text forward in the same process stays bit\-identical; \(iii\) gradient isolation \(only adapter and connector parameters receive gradients\); \(iv\) checkpoint resume round\-trips adapters and refuses adapter\-presence mismatches; \(v\) warm\-start semantics from a connector\-only checkpoint; \(vi\) a gate that fails to span the re\-run forwards of gradient checkpointing raisesCheckpointErrorloudly\. The training loop assertsbase\_drift=0=0\(maximum absolute change over every base parameter\) after every training run\. The release smoke test loads the packaged checkpoint from the public repository and embeds identical text and image inputs twice, once with the adapter hooks attached \(gate closed\) and once with every hook removed, requiringtorch\.equalon both modalities; it also exercises the audio path end to end through the public loading code\. The gate contract is enforced at the entry points on both sides of the release:encode\_textin the training code andembed\_imagein the released inference code \(the vision path shares the hooked decoder layers\) each raise if called under an open gate, so neither cached text targets nor image embeddings can be silently adapted\.

## Appendix CVariants and checks not run

In the spirit of reporting what was and was not tested: the rank sweep \(128, 384\) and the fine\-tune\-length check \(a 200\-step arm scored 0\.273/0\.695 against 0\.302/0\.743 at 400 steps under the release\-protocol rescore of its lineage; a 600\-step arm on the predecessor showed the other side of the optimum\) are the ablations we have\. Not run: adapters on layer subsets, randomly initialized \(non\-zero\) adapters, a trained ungated\-adapter arm \(its inference\-time effect on non\-audio inputs is what Table[4](https://arxiv.org/html/2609.26182#S4.T4)’s open\-gate row makes visible\), and multi\-seed replications\. None of these is blocked by the design; they were out of budget, and we state them rather than imply them\.

## Appendix DHyperparameters

Connector: perceiver\-resampler, internal width 384, 64 latent queries, 6 blocks, 16\.4M parameters\. Adapters \(Eq\.[1](https://arxiv.org/html/2609.26182#S3.E1)\): rank 384, SiLU, LayerNorm\-first bottleneck, zero\-init up\-projection, fp32 compute cast back to stream dtype, 44\.2M parameters over 28 layers\. Loss: symmetric InfoNCE at every Matryoshka rung with a light covariance penalty; audio\-to\-text negatives augmented with the full\-corpus frozen\-text bank; soft labels \(β=0\.3\\beta\{=\}0\.3\) and false\-negative masking \(τ=0\.98\\tau\{=\}0\.98\) at 500K\+ scale\. Optimizer AdamW, cosine decay, 5% warmup\. Training input format is the base’s native chat template; the format study and full recipe details are in the companion technical report\(Tonmoy et al\.,[2026](https://arxiv.org/html/2609.26182#bib.bib29)\)\.

## References

- Alayrac et al\. \(2022\)Jean\-Baptiste Alayrac, Jeff Donahue, Pauline Luc, Antoine Miech, Iain Barr, Yana Hasson, Karel Lenc, Arthur Mensch, Katherine Millican, Malcolm Reynolds, et al\. 2022\.Flamingo: a visual language model for few\-shot learning\.*Advances in Neural Information Processing Systems \(NeurIPS\)*\.ArXiv:2204\.14198\.
- Bachlechner et al\. \(2021\)Thomas Bachlechner, Bodhisattwa Prasad Majumder, Huanru Henry Mao, Garrison W\. Cottrell, and Julian McAuley\. 2021\.ReZero is all you need: Fast convergence at large depth\.In*Uncertainty in Artificial Intelligence \(UAI\)*\.ArXiv:2003\.04887\.
- Bao et al\. \(2022\)Hangbo Bao, Wenhui Wang, Li Dong, Qiang Liu, Owais Khan Mohammed, Kriti Aggarwal, Subhojit Som, and Furu Wei\. 2022\.VLMo: Unified vision\-language pre\-training with mixture\-of\-modality\-experts\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.ArXiv:2111\.02358\.
- Chen et al\. \(2020\)Honglie Chen, Weidi Xie, Andrea Vedaldi, and Andrew Zisserman\. 2020\.Vggsound: A large\-scale audio\-visual dataset\.In*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*\.ArXiv:2004\.14368\.
- Chen et al\. \(2016\)Tianqi Chen, Ian Goodfellow, and Jonathon Shlens\. 2016\.Net2Net: Accelerating learning via knowledge transfer\.In*International Conference on Learning Representations \(ICLR\)*\.ArXiv:1511\.05641\.
- Dinkel et al\. \(2026\)Heinrich Dinkel, Zhiyong Yan, Tianzi Wang, Yongqing Wang, Xingwei Sun, Yadong Niu, Jizhong Liu, Gang Li, Junbo Zhang, and Jian Luan\. 2026\.Glap: General contrastive audio\-text pretraining across domains and languages\.In*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*\.ArXiv:2506\.11350\.
- Dinkel et al\. \(2024\)Heinrich Dinkel, Zhiyong Yan, Yongqing Wang, Junbo Zhang, Yujun Wang, and Bin Wang\. 2024\.Scaling up masked audio encoder learning for general audio classification\.In*Proceedings of Interspeech*\.ArXiv:2406\.06992\.
- Drossos et al\. \(2020\)Konstantinos Drossos, Samuel Lipping, and Tuomas Virtanen\. 2020\.Clotho: an audio captioning dataset\.In*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*\.ArXiv:1910\.09387\. Evaluation split v2\.1 from Zenodo record 4783391\.
- Girdhar et al\. \(2023\)Rohit Girdhar, Alaaeldin El\-Nouby, Zhuang Liu, Mannat Singh, Kalyan Vasudev Alwala, Armand Joulin, and Ishan Misra\. 2023\.Imagebind: One embedding space to bind them all\.In*IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*\.ArXiv:2305\.05665\.
- Gong et al\. \(2023\)Yuan Gong, Sameer Khurana, Leonid Karlinsky, and James Glass\. 2023\.Whisper\-at: Noise\-robust automatic speech recognizers are also strong general audio event taggers\.In*Proceedings of Interspeech*\.ArXiv:2307\.03183\.
- Hönicke et al\. \(2026\)Florian Hönicke, Michael Günther, Andreas Koukounas, Mohammad Kalim Akram, Scott Martens, Saba Sturua, and Han Xiao\. 2026\.jina\-embeddings\-v5\-omni: Geometry\-preserving embeddings via locked aligned towers\.*arXiv preprint arXiv:2605\.08384*\.
- Houlsby et al\. \(2019\)Neil Houlsby, Andrei Giurgiu, Stanislaw Jastrzebski, Bruna Morrone, Quentin de Laroussilhe, Andrea Gesmundo, Mona Attariyan, and Sylvain Gelly\. 2019\.Parameter\-efficient transfer learning for NLP\.In*Proceedings of the 36th International Conference on Machine Learning \(ICML\)*\.ArXiv:1902\.00751\.
- Hu et al\. \(2022\)Edward J\. Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, and Weizhu Chen\. 2022\.LoRA: Low\-rank adaptation of large language models\.In*International Conference on Learning Representations \(ICLR\)*\.ArXiv:2106\.09685\.
- Jia et al\. \(2021\)Xinyu Jia, Chuang Zhu, Minzhen Li, Wenqi Tang, Shengjie Liu, and Wenli Zhou\. 2021\.LLVIP: A visible\-infrared paired dataset for low\-light vision\.In*IEEE/CVF International Conference on Computer Vision \(ICCV\) Workshops*\.ArXiv:2108\.10831\.
- Kim et al\. \(2019\)Chris Dongjoo Kim, Byeongchang Kim, Hyunmin Lee, and Gunhee Kim\. 2019\.Audiocaps: Generating captions for audios in the wild\.In*Proceedings of NAACL\-HLT*\.
- Kusupati et al\. \(2022\)Aditya Kusupati, Gantavya Bhatt, Aniket Rege, Matthew Wallingford, Aditya Sinha, Vivek Ramanujan, William Howard\-Snyder, Kaifeng Chen, Sham Kakade, Prateek Jain, and Ali Farhadi\. 2022\.Matryoshka representation learning\.In*Advances in Neural Information Processing Systems \(NeurIPS\)*\.ArXiv:2205\.13147\.
- Li et al\. \(2023\)Junnan Li, Dongxu Li, Silvio Savarese, and Steven Hoi\. 2023\.BLIP\-2: Bootstrapping language\-image pre\-training with frozen image encoders and large language models\.In*Proceedings of the 40th International Conference on Machine Learning \(ICML\)*\.ArXiv:2301\.12597\.
- Li et al\. \(2026\)Mingxin Li, Yanzhao Zhang, Dingkun Long, Keqin Chen, Sibo Song, Shuai Bai, Zhibo Yang, Pengjun Xie, An Yang, Dayiheng Liu, Jingren Zhou, and Junyang Lin\. 2026\.Qwen3\-vl\-embedding and qwen3\-vl\-reranker: A unified framework for state\-of\-the\-art multimodal retrieval and ranking\.*arXiv preprint arXiv:2601\.04720*\.
- Liu et al\. \(2024\)Jizhong Liu, Gang Li, Junbo Zhang, Heinrich Dinkel, Yongqing Wang, Zhiyong Yan, Yujun Wang, and Bin Wang\. 2024\.Enhancing automated audio captioning via large language models with optimized audio encoding\.In*Proceedings of Interspeech*\.ArXiv:2406\.13275\.
- Mei et al\. \(2024\)Xinhao Mei, Chutong Meng, Haohe Liu, Qiuqiang Kong, Tom Ko, Chengqi Zhao, Mark D\. Plumbley, Yuexian Zou, and Wenwu Wang\. 2024\.Wavcaps: A chatgpt\-assisted weakly\-labelled audio captioning dataset for audio\-language multimodal research\.*IEEE/ACM Transactions on Audio, Speech, and Language Processing*\.ArXiv:2303\.17395\.
- Meng et al\. \(2025\)Rui Meng et al\. 2025\.VLM2Vec\-V2: Advancing multimodal embedding for videos, images, and visual documents\.*arXiv preprint arXiv:2507\.04590*\.The MMEB\-V2 benchmark\.
- Niizumi et al\. \(2025\)Daisuke Niizumi, Daiki Takeuchi, Yasunori Ohishi, Noboru Harada, and Kunio Kashino\. 2025\.M2d\-clap: Exploring general\-purpose audio\-language representations beyond clap\.*arXiv preprint arXiv:2503\.22104*\.
- Panagopoulou et al\. \(2024\)Artemis Panagopoulou, Le Xue, Ning Yu, Junnan Li, Dongxu Li, Shafiq Joty, Ran Xu, Silvio Savarese, Caiming Xiong, and Juan Carlos Niebles\. 2024\.X\-InstructBLIP: A framework for aligning x\-modal instruction\-aware representations to LLMs and emergent cross\-modal reasoning\.In*European Conference on Computer Vision \(ECCV\)*\.ArXiv:2311\.18799\.
- Piczak \(2015\)Karol J\. Piczak\. 2015\.Esc: Dataset for environmental sound classification\.In*Proceedings of the 23rd ACM International Conference on Multimedia*\.
- Radford et al\. \(2023\)Alec Radford, Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, and Ilya Sutskever\. 2023\.Robust speech recognition via large\-scale weak supervision\.In*Proceedings of the 40th International Conference on Machine Learning \(ICML\)*\.ArXiv:2212\.04356\.
- Rusu et al\. \(2016\)Andrei A\. Rusu, Neil C\. Rabinowitz, Guillaume Desjardins, Hubert Soyer, James Kirkpatrick, Koray Kavukcuoglu, Razvan Pascanu, and Raia Hadsell\. 2016\.Progressive neural networks\.*arXiv preprint arXiv:1606\.04671*\.
- Sheng et al\. \(2024\)Ying Sheng, Shiyi Cao, Dacheng Li, Coleman Hooper, Nicholas Lee, Shuo Yang, Christopher Chou, Banghua Zhu, Lianmin Zheng, Kurt Keutzer, Joseph E\. Gonzalez, and Ion Stoica\. 2024\.S\-LoRA: Serving thousands of concurrent LoRA adapters\.In*Proceedings of Machine Learning and Systems \(MLSys\)*\.ArXiv:2311\.03285\.
- Tonmoy \(2026\)Abdul Basit Tonmoy\. 2026\.Discriminative axis, not data volume: What a contrastive corpus teaches an audio embedding\.*arXiv preprint arXiv:2608\.01560*\.
- Tonmoy et al\. \(2026\)Abdul Basit Tonmoy, Kazi Fardinul Hoque, Md\. Shahrier Islam Arham, and Arman Luthra\. 2026\.Fusion embedding: A unified embedding space for text, image, video, and audio\.*arXiv preprint arXiv:2607\.18666*\.
- van den Oord et al\. \(2018\)Aäron van den Oord, Yazhe Li, and Oriol Vinyals\. 2018\.Representation learning with contrastive predictive coding\.*arXiv preprint arXiv:1807\.03748*\.
- Wang et al\. \(2023\)Peng Wang, Shijie Wang, Junyang Lin, Shuai Bai, Xiaohuan Zhou, Jingren Zhou, Xinggang Wang, and Chang Zhou\. 2023\.ONE\-PEACE: Exploring one general representation model toward unlimited modalities\.*arXiv preprint arXiv:2305\.11172*\.
- Wang et al\. \(2024\)Zehan Wang, Ziang Zhang, Xize Cheng, Rongjie Huang, Luping Liu, Zhenhui Ye, Haifeng Huang, Yang Zhao, Tao Jin, Peng Gao, and Zhou Zhao\. 2024\.Freebind: Free lunch in unified multimodal space via knowledge fusion\.In*International Conference on Machine Learning \(ICML\)*\.ArXiv:2405\.04883\.
- Wang et al\. \(2025\)Zehan Wang, Ziang Zhang, Minjie Hong, Hang Zhang, Luping Liu, Rongjie Huang, Xize Cheng, Shengpeng Ji, Tao Jin, Hengshuang Zhao, and Zhou Zhao\. 2025\.Omnibind: Large\-scale omni multimodal representation via binding spaces\.In*International Conference on Learning Representations \(ICLR\)*\.ArXiv:2407\.11895\.
- Wu et al\. \(2024\)Chengyue Wu et al\. 2024\.LLaMA pro: Progressive LLaMA with block expansion\.In*Proceedings of the Annual Meeting of the Association for Computational Linguistics \(ACL\)*\.ArXiv:2401\.02415\.
- Wu et al\. \(2023\)Yusong Wu, Ke Chen, Tianyu Zhang, Yuchen Hui, Taylor Berg\-Kirkpatrick, and Shlomo Dubnov\. 2023\.Large\-scale contrastive language\-audio pretraining with feature fusion and keyword\-to\-caption augmentation\.In*IEEE International Conference on Acoustics, Speech and Signal Processing \(ICASSP\)*\.ArXiv:2211\.06687\.
- Xu et al\. \(2026\)Jilan Xu, Carl Thomé, Danijela Horak, Weidi Xie, and Andrew Zisserman\. 2026\.Scaling audio\-text retrieval with multimodal large language models\.*arXiv preprint arXiv:2602\.18010*\.
- Xu et al\. \(2025\)Jin Xu et al\. 2025\.Qwen2\.5\-omni technical report\.*arXiv preprint arXiv:2503\.20215*\.
- Yoo et al\. \(2026\)HaeJun Yoo, Yongseop Shin, Insung Lee, Myoung\-Wan Koo, and Du\-Seong Chang\. 2026\.Omni\-embed\-audio: Leveraging multimodal llms for robust audio\-text retrieval\.In*Proceedings of the Annual Meeting of the Association for Computational Linguistics \(ACL\)*\.ArXiv:2604\.18360\.
- Zhang et al\. \(2020\)Jeffrey O\. Zhang, Alexander Sax, Amir Zamir, Leonidas Guibas, and Jitendra Malik\. 2020\.Side\-tuning: A baseline for network adaptation via additive side networks\.In*European Conference on Computer Vision \(ECCV\)*\.ArXiv:1912\.13503\.