One Domain, Many Tongues: Composing Domain and Language LoRAs for Cross-Lingual Remote-Sensing MLLMs without Paired Data

# Composing Domain and Language LoRAs for Cross-Lingual Remote-Sensing MLLMs without Paired Data
Source: [https://arxiv.org/html/2609.26097](https://arxiv.org/html/2609.26097)
###### Abstract

Remote\-sensing \(RS\) multimodal large language models \(MLLMs\) are trained and evaluated only in English, while text\-only instruction data covers over 100 languages\. We proposeModl\(*M*utually*O*rthogonal*D*omain–*L*anguage composition\), a recipe that adds new languages to an English RS MLLM without a single multilingual RS example: a domain LoRA trained on English RS imagery and a language LoRA trained on text alone are learned jointly, under one loss term that keeps the two updates mutually orthogonal at every layer throughout training\. This constraint is the recipe’s active ingredient\. Without it, the same training answers RS questions correctly*but in English*, erases much of the base model’s multilingual text ability, and diverges on one seed in three; sixteen alternatives, from training\-free merging to prior orthogonality variants, fail the same way\.Modlrepairs every failure on every seed: answers are correct and in the target language 56–71% of the time, where the best alternative reaches 27% and most stay below 8%, text ability stays at the level of the untrained base, and on Spanish it surpasses Qwen2\.5\-VL\-7B, with zero multilingual–multimodal data\. A single five\-language adapter retains English, Spanish, and Vietnamese at full strength across three seeds; non\-Latin scripts remain an open boundary\.111Code, adapters, and the preregistered evaluation protocol will be released upon publication\.

One Domain, Many Tongues: Composing Domain and Language LoRAs for Cross\-Lingual Remote\-Sensing MLLMs without Paired Data

Xuechen LiUniversity of Minnesota, Twin Citiesli003487@umn\.edu

## 1Introduction

![Refer to caption](https://arxiv.org/html/2609.26097v1/x1.png)Figure 1:One loss term changes everything\.Left: correct\-and\-in\-language accuracy \(A∩\\capL\) on the GeoChat\-Bench classification holdout \(2,195 items per language, es/vi average\)\.Right: text\-only multilingual competence, measured as likelihood\-ranked Belebele accuracy \(900 items per language, es/vi average\), which is immune to generation\-side defects\. Bars are means over three seeds; black dots mark individual seeds; the dashed line is the Qwen2\.5\-VL\-7B reference\. One mutual\-orthogonality loss term \(Modl, green\) separates the only working system from every alternative\. The highJointseed dot on the left is the diverged seed’s step\-500 substitute \(App\.[D](https://arxiv.org/html/2609.26097#A4)\)\.Remote\-sensing \(RS\) multimodal large language models \(MLLMs\) are built and evaluated in English\(Kuckreja et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib18); Muhtar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib26); Zhang et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib54); Soni et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib37)\)\. The standard route to multilinguality trains on millions of machine\-translated \(MT\) multimodal instructions\(Rasheed et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib34); Yue et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib51); Geigle et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib11)\); transplanted to a vertical domain, its cost multiplies by the number of languages\.222The only partly non\-English RS resources are English–Chinese\(Zhou et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib57); Wang et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib40)\); no RS resource covers a low\-resource language\.The modular parameter\-efficient fine\-tuning \(PEFT\) literature promises a far cheaper route: train a*domain*LoRA \(low\-rank adaptation;Hu et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib13)\) on English RS instructions and a*language*LoRA on text\-only instructions\(Li et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib21)\), and compose them\(Pfeiffer et al\.,[2020](https://arxiv.org/html/2609.26097#bib.bib30); Ansell et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib1); Chronopoulou et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib4); Bandarkar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib3)\)\. That literature, however, is text\-only, and two obstacles stand between it and the multimodal setting: LLaVA\-style models drift to English the moment an image enters the context \(image\-induced fidelity loss, IFL;Hinck et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib12)\), and independently trained LoRA updates interfere when summed\(Stoica et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib38)\)\.

We proposeModl: joint training of the two adapters under a symmetric*mutual\-orthogonality*constraint, a single loss term that keeps the domain and language updates in disjoint subspaces at every layer throughout training\. We validateModlin the first systematic, preregistered study of domain×\\timeslanguage LoRA composition in an MLLM: LLaVA\-1\.5\-7B, five typologically diverse languages, a matched training budget for every system, three seeds, and decision rules frozen in advance\. The comparison spans every family of alternatives: plain joint training, fourteen training\-free mergers, prior orthogonal\-training methods \(OSRM, O\-LoRA\), andFic, a function\-space consistency variant representing the behavioral alternative suggested by recent skepticism that geometry is what controls interference\(Zhang et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib52)\)\.

The comparison is one\-sided \(Fig\.[1](https://arxiv.org/html/2609.26097#S1.F1)\)\. Capability composes but language does not: naively co\-trained adapters answer Spanish and Vietnamese RS questions correctly*in English*, a fidelity failure that scoring against translated references mistakes for incapacity\. Naive joint training is also destructive: it erases the base model’s text\-only multilingual competence, degrades even its native target\-language RS accuracy, and diverges on one of three seeds\. Among the alternatives,Ficrepairs only the text\-competence loss\.Modl, one loss term, repairs everything, on every seed and across two orders of magnitude of its single hyperparameter; on Spanish scene classification it surpasses even Qwen2\.5\-VL\-7B’s incidental multilinguality in correct\-and\-in\-language accuracy \(A∩\\capL\)\.

Contributions:

1. 1\.The first cross\-lingual RS MLLM recipe: mutually orthogonal joint training of a domain and a language LoRA, the only trained system that beats the untrained base on A∩\\capL \(§[5](https://arxiv.org/html/2609.26097#S5)\)\.
2. 2\.A diagnosis that reframes the problem: the obstacle to composition is interference and output\-language fidelity, not capability transfer \(§[5](https://arxiv.org/html/2609.26097#S5)\)\.
3. 3\.Mechanism evidence: under one matched budget, a function\-space variant with the same goal \(Fic\) and prior initialization\-time and sequential orthogonality \(OSRM, O\-LoRA\) all fail\. What matters is that the constraint is mutual and active throughout training, contrary to text\-only evidence on orthogonality\(Zhang et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib52)\)\(§[3\.4](https://arxiv.org/html/2609.26097#S3.SS4)\)\.
4. 4\.A preregistered protocol with two quantified evaluation pitfalls\(missing end\-of\-sequence \(EOS\) supervision,7×7\\timesmetric corruption; inconsistently translated references, up to 41% of items unscorable\) and corrected class\-identity scoring \(§[4](https://arxiv.org/html/2609.26097#S4)\)\.

## 2Related Work

#### Remote\-sensing MLLMs\.

GeoChat\(Kuckreja et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib18)\)established LoRA\-tuning of a LLaVA\-family model\(Liu et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib24)\)on RS instructions, followed by RSGPT\(Hu et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib15)\), LHRS\-Bot\(Muhtar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib26)\), EarthGPT\(Zhang et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib54)\), EarthDial\(Soni et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib37)\), and many successors\. All are English\-only\. Multilingual RS resources are embryonic: EN–ZH captions and annotations \(BRSIC, XLRS\-Bench;Zhou et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib57); Wang et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib40)\), the RS\-M\-CLIP encoder\(Silva et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib36)\), retrieval\-based training\-free captioning\(Rebelo et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib35)\), and translation\-as\-augmentation for RSVQA\(Yuan et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib50)\)\. None yields an instruction\-following RS assistant in any non\-English language\.

#### Multilingual MLLMs and IFL\.

PALO\(Rasheed et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib34)\), mBLIP\(Geigle et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib10)\), Pangea\(Yue et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib51)\), and Centurio\(Geigle et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib11)\)train on multilingual multimodal data; VisCPM\(Hu et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib14)\)exploits a multilingual backbone;Pikabea et al\. \([2025](https://arxiv.org/html/2609.26097#bib.bib31)\)inject text\-only multilingual data*during*visual supervised fine\-tuning \(SFT\) to mitigate IFL\(Hinck et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib12)\); Aya Vision\(Dash et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib8)\)merges to*preserve*pre\-existing multilinguality\. None adds a new language to an English\-centric MLLM by composing a text\-only language module, and none addresses a specialized visual domain\.

#### Modular composition for cross\-lingual transfer\.

MAD\-X\(Pfeiffer et al\.,[2020](https://arxiv.org/html/2609.26097#bib.bib30)\)and LT\-SFT\(Ansell et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib1)\)compose language and task modules in encoder models; the LLM\-era line includes language/task arithmetic\(Chronopoulou et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib4)\), AdaMergeX\(Zhao et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib55)\), expert layer swapping\(Bandarkar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib3)\), and language–task LoRA interpolation\(Lee et al\.,[2025a](https://arxiv.org/html/2609.26097#bib.bib19)\)\. xGQA\(Pfeiffer et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib29)\)stacked adapters for cross\-lingual visual question answering \(VQA\) in pre\-LLM models\. All of this is text\-only or pre\-LLM; the multimodal vertical\-domain instantiation is our subject\.

#### Merging, merge\-aware training, and function\-space fusion\.

Training\-free mergers reduce interference at merge time: task arithmetic\(Ilharco et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib16)\), TIES\(Yadav et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib45)\), DARE\(Yu et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib49)\), AdaMerging’s layer\-wise coefficients\(Yang et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib47)\), KnOTS’ SVD alignment\(Stoica et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib38)\), and CAT’s learned concatenation\(Prabhakar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib33)\); surveys inYang et al\. \([2026](https://arxiv.org/html/2609.26097#bib.bib46)\)\. In MLLMs, merging composes heterogeneous models\(Du et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib9)\)and transfers multilinguality from a multilingual LLM’s residuals\(Wang et al\.,[2026](https://arxiv.org/html/2609.26097#bib.bib43)\)\. Merge\-*aware training*makes models composable by construction: fixed orthogonal bases \(orthogonal adaptation, QR\-LoRA;Po et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib32); Yang et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib48); Liu et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib25)\), data\-aware pre\-fine\-tuning constraints\(Zhang and Zhou,[2025](https://arxiv.org/html/2609.26097#bib.bib53)\), sharpness\-aware fine\-tuning\(Lee et al\.,[2025b](https://arxiv.org/html/2609.26097#bib.bib20)\), linearized fine\-tuning\(Ortiz\-Jimenez et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib27)\), and sequential inter\-LoRA orthogonality for continual learning\(Wang et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib42)\); all are parameter\-geometric\.Zhang et al\. \([2025](https://arxiv.org/html/2609.26097#bib.bib52)\)report that strict inter\-LoRA orthogonality alone does not deliver compositionality in text\-only merging; our multimodal evidence points the other way, and sharpens*which*orthogonality matters: we compare initialization\-time \(OSRM\), sequential \(O\-LoRA\), and our*mutual, simultaneous*constraint under one budget, and only the last restores language fidelity \(Table[1](https://arxiv.org/html/2609.26097#S4.T1)\); the prior families reproduce the naive\-joint failure signature\.Ficinstead belongs to the*function\-space*family: RegMean matches linear\-layer outputs at merge time\(Jin et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib17)\), FuseLLM distills several source LLMs into one via continual training\(Wan et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib39)\), and multi\-teacher distillation has been used to repair merged models post hoc\(Dalili and Mahdavi,[2025](https://arxiv.org/html/2609.26097#bib.bib6)\); Learning\-without\- Forgetting\-style consistency preserves old behavior in continual learning\(Li and Hoiem,[2017](https://arxiv.org/html/2609.26097#bib.bib23)\)\.Ficdiffers from all of these in that the consistency is imposed*during*adapter training, on the model’s*own*unpaired training streams, with the detached single\-adapter configuration as teacher \(no transfer set, no separate distillation phase, no external teacher\), and in that the composed axes are domain and language in a multimodal LLM\.

## 3Method

### 3\.1Problem Setup

LetfW0f\_\{W\_\{0\}\}be an English\-centric MLLM \(LLaVA\-1\.5\-7B\(Liu et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib24)\): CLIP ViT\-L/336, projector, Vicuna\-7B\) with parametersW0W\_\{0\}\. The English\-centric backbone is a deliberate control: it isolates the language\-acquisition effect that a strongly multilingual backbone would confound \(we include such a backbone as a reference point, §[4](https://arxiv.org/html/2609.26097#S4)\)\. We are given \(i\) an English RS instruction corpus𝒟dom\\mathcal\{D\}\_\{\\mathrm\{dom\}\}\(image–instruction–response triples\) and \(ii\) per target languageℓ∈ℒ\\ell\\in\\mathcal\{L\}a*text\-only*instruction corpus𝒟ℓ\\mathcal\{D\}\_\{\\ell\}with no RS content and no images\. The goal is a single model that answers RS questions posed inℓ\\ell,*in*ℓ\\ell, having never seen an RS example inℓ\\ellor anyℓ\\ell\-language image–text pair, with no test\-time routing, language detection, or per\-input adapter switching\.

### 3\.2Domain and Language LoRAs

Each adapted weightW0\(m\)∈ℝd×kW^\{\(m\)\}\_\{0\}\\in\\mathbb\{R\}^\{d\\times k\}receives LoRA\(Hu et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib13)\)updatesΔ​W\(m\)=αrr​B\(m\)​A\(m\)\\Delta W^\{\(m\)\}=\\tfrac\{\\alpha\_\{r\}\}\{r\}B^\{\(m\)\}A^\{\(m\)\}\. TheD\-LoRA\{Δ​WD\(m\)\}\\\{\\Delta W\_\{D\}^\{\(m\)\}\\\}trains on𝒟dom\\mathcal\{D\}\_\{\\mathrm\{dom\}\}and attaches to attention and MLP projections of all LLM layers and to the vision–language projector\. EachL\-LoRA\{Δ​WLℓ\(m\)\}\\\{\\Delta W\_\{L\_\{\\ell\}\}^\{\(m\)\}\\\}trains on𝒟ℓ\\mathcal\{D\}\_\{\\ell\}\(no images\) and attaches to the LLM only, because a text\-only corpus provides no signal for visual alignment\. At inference the two updates are simply added into the base weights \(§[3\.5](https://arxiv.org/html/2609.26097#S3.SS5)\)\.

### 3\.3Joint Training and the Geometric Hypothesis

Training alternates micro\-batches from𝒟dom\\mathcal\{D\}\_\{\\mathrm\{dom\}\}\(updatingθD\\theta\_\{D\}\) and from the language corpora \(updatingθL\\theta\_\{L\}\); within a run the target languages share a singleL\-LoRAtrained on their union \(es\+vi in the core setting, one language in per\-pair runs, five in the five\-way variant\), soD\-LoRAis optimized to coexist with the language subspace it will be deployed with\. We call this plain schemeJoint\. It already differs from independent training \(the input to all merging baselines\) by letting the optimizer see both objectives, though never a paired example\.

The merge\-aware\-training literature predicts that interference should be removed by*decoupling parameter subspaces*\. Prior instantiations constrain sequential task streams\(Wang et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib42)\), initialization\-time subspaces\(Zhang and Zhou,[2025](https://arxiv.org/html/2609.26097#bib.bib53)\), or fixed bases\(Po et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib32)\); none address two heterogeneous modules trained simultaneously\. Our instantiation,Modl, adds a symmetric*mutual*penalty on row/column\-normalized factors of the two adapters at every layer,

ℛ⟂\(m\)=‖A^D\(m\)​A^Lℓ\(m\)⊤‖F2\+‖B^D\(m\)⊤​B^Lℓ\(m\)‖F2,\\mathcal\{R\}\_\{\\perp\}^\{\(m\)\}=\\big\\\|\\hat\{A\}\_\{D\}^\{\(m\)\}\\hat\{A\}\_\{L\_\{\\ell\}\}^\{\(m\)\\top\}\\big\\\|\_\{F\}^\{2\}\+\\big\\\|\\hat\{B\}\_\{D\}^\{\(m\)\\top\}\\hat\{B\}\_\{L\_\{\\ell\}\}^\{\(m\)\}\\big\\\|\_\{F\}^\{2\},\(1\)drivingΔ​WD⊤​Δ​WL≈0\\Delta W\_\{D\}^\{\\top\}\\Delta W\_\{L\}\\\!\\approx\\\!0andΔ​WD​Δ​WL⊤≈0\\Delta W\_\{D\}\\Delta W\_\{L\}^\{\\top\}\\\!\\approx\\\!0, weighted byλ⟂\\lambda\_\{\\perp\}\(0\.1 primary; results are stable acrossλ⟂∈\[0\.01,1\]\\lambda\_\{\\perp\}\\\!\\in\\\!\[0\.01,1\]\) and trained identically toJointotherwise\.Modlis the paper’s proposed recipe; Figure[2](https://arxiv.org/html/2609.26097#S3.F2)sketches the training setup and the three failure modes the constraint removes\. To test whether its effect is genuinely geometric, we also construct the natural function\-space alternative,Ficbelow, which pursues the same non\-interference goal without constraining where the modules live; the contrast between the two carries the mechanistic claim\. \(The preregistration originally favored the functional hypothesis; the reversal is documented in App\.[B](https://arxiv.org/html/2609.26097#A2)\.\)

![Refer to caption](https://arxiv.org/html/2609.26097v1/figs/fig2_pipeline.png)Figure 2:TheModlrecipe, shown on real training and test examples\.Left: the two unpaired training streams as they actually appear to the model: an English RS instruction with its image \(supervisingD\-LoRAonly\) and a text\-only Spanish instruction with no image \(supervisingL\-LoRAonly\); targets end with the supervised EOS\.Center: both adapters are trained jointly on the frozen backbone; the mutual orthogonality penaltyℛ⟂\\mathcal\{R\}\_\{\\perp\}\(Eq\.[1](https://arxiv.org/html/2609.26097#S3.E1)\) keeps the two updates in disjoint subspaces at every layer and every step\.Right: deployment simply adds both updates,W0\+Δ​WD\+Δ​WLW\_\{0\}\+\\Delta W\_\{\\mathrm\{D\}\}\+\\Delta W\_\{\\mathrm\{L\}\}, and is queried on a combination never seen in training — a Vietnamese question about an RS image\. Without the constraint \(Joint\) the model names the correct class in English; with it \(Modl\) it answers correctly in Vietnamese\. The footer notes the other two repairs: multilingual text retention \(Belebele\) and optimization stability \(no constrained run diverged, vs\. 3 of 7 for the naive recipe\)\.
### 3\.4Fic: Functional Interference Consistency

What we actually require of composition is behavioral, not geometric:*on domain inputs, adding the language module must not change what the domain module does; on language inputs, vice versa*\.Ficimposes exactly this, on the two unpaired streams already present in joint training\.

For a micro\-batch from𝒟dom\\mathcal\{D\}\_\{\\mathrm\{dom\}\}, let the*teacher*be the model with only the domain adapter active,pT=fW0\+Δ​WDp\_\{T\}=f\_\{W\_\{0\}\+\\Delta W\_\{D\}\}, with gradients stopped, and the*student*be the additive compositionpS=fW0\+Δ​WD\+Δ​WLℓp\_\{S\}=f\_\{W\_\{0\}\+\\Delta W\_\{D\}\+\\Delta W\_\{L\_\{\\ell\}\}\}\. WithSSthe set of supervised \(response\) token positions, capped atKKper micro\-step,

ℒFicdom=1\|S\|∑t∈SKL\(pT\(⋅\|x,y<t\)∥pS\(⋅\|x,y<t\)\),\\mathcal\{L\}\_\{\\textsc\{Fic\}\}^\{\\mathrm\{dom\}\}=\\tfrac\{1\}\{\|S\|\}\\textstyle\\sum\_\{t\\in S\}\\mathrm\{KL\}\\\!\\left\(p\_\{T\}\(\\cdot\|x,y\_\{<t\}\)\\,\\\|\\,p\_\{S\}\(\\cdot\|x,y\_\{<t\}\)\\right\),\(2\)at temperatureτ\\tau\. Symmetrically, on a batch from𝒟ℓ\\mathcal\{D\}\_\{\\ell\}the teacher is the language\-only modelfW0\+Δ​WLℓf\_\{W\_\{0\}\+\\Delta W\_\{L\_\{\\ell\}\}\}and the student is the same composition\. The total objective per stream is the task cross\-entropy of that stream’s own adapter plusλFic​ℒFic\\lambda\_\{\\textsc\{Fic\}\}\\,\\mathcal\{L\}\_\{\\textsc\{Fic\}\}\. Frozen configuration from the preregistered pilot:λFic=0\.5\\lambda\_\{\\textsc\{Fic\}\}\{=\}0\.5,τ=1\.0\\tau\{=\}1\.0,K=64K\{=\}64, applied from the first step with no warm\-up or ramp\.

Three properties matter\.*\(i\) No pairing:*the two consistency terms each use only their own stream; no multilingual RS example, real or synthetic, is ever constructed\.*\(ii\) No routing:*the deployed model is the plain additive composition;Ficshapes training, not inference\.*\(iii\) Nearly free:*the teacher is the student’s own detached sub\-configuration: one extra partial forward on at mostKKsupervised tokens per micro\-step, no external models, no transfer set, no separate distillation phase \(contrast RegMean’s merge\-time solves\(Jin et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib17)\), FuseLLM’s continual distillation corpus\(Wan et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib39)\), and post\-hoc multi\-teacher repair\(Dalili and Mahdavi,[2025](https://arxiv.org/html/2609.26097#bib.bib6)\)\)\. Conceptually,Ficis the functional projection of the orthogonality desideratum: rather than demandingΔ​WL\\Delta W\_\{L\}live in a subspace where it*cannot*affect domain computation anywhere, it demandsΔ​WL\\Delta W\_\{L\}*happen not to*affect domain behavior on the domain distribution, seemingly the weaker, better\-targeted condition\. Our results show the opposite: the ostensibly over\-strong subspace condition is what protects language fidelity and the base model’s competences, while the distribution\-limited functional condition leaves both exposed\.

### 3\.5Composition at Deployment

Default deployment is uniform additionW=W0\+Δ​WD\+Δ​WLℓW=W\_\{0\}\+\\Delta W\_\{D\}\+\\Delta W\_\{L\_\{\\ell\}\}with no tuned coefficients \(*route\-free CAT*in our runs\); a module\-wise asymmetric variant motivated by the upper\-layer concentration of language identity\(Bandarkar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib3); Hinck et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib12)\)was cut with the other reduced\-scope items \(§[4](https://arxiv.org/html/2609.26097#S4)\) and is future work\.

## 4Experimental Setup

#### Models, data, budget\.

Base: LLaVA\-1\.5\-7B\.D\-LoRA: rank 64, on 100k stratified GeoChat\-Instruct\(Kuckreja et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib18)\)\.L\-LoRAℓ: rank 32, on 67k Bactrian\-X\(Li et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib21)\)forℓ∈\\ell\\in\{es, ar, hi, vi, sw\} \(four scripts; Vietnamese and Swahili low\-resource\)\.Compute\-bounded protocol:every trained system, including all baselines, receives an identical matched budget of 1,000 optimizer steps \(a preregistered fallback from 2,500, executed before any run passed step 100 when measured A100 throughput made the larger budget infeasible in the reserved window\) with identical data order, token budget, and stopping rule; core comparisons use 3 seeds; hyperparameters were frozen from a preregistered es/vi pilot before any test\-set contact\. We report GPU\-hours for every system\. Claims are explicitly scoped to this budget; the full\-epoch regime is future work\.

Table 1:Main results on the GeoChat\-Bench classification holdout \(2,195 items/language\) and Belebele likelihood retention\.Joint,Fic, andModlare means over three seeds \(J0 seed 44 diverged; its last healthy checkpoint, step 500, is substituted and the diverged run reported in App\.[D](https://arxiv.org/html/2609.26097#A4)\); the merge, OSRM, andJoint2×\\timesrows are single runs, as their scores sit at a floor no seed resample plausibly escapes\. Acc = class\-identity accuracy \(any accepted surface form\); LF = response in the query language; A∩\\capL = correct and in the query language\.Modlis the only matched\-budget system that beats the untrained base on A∩\\capL, and it exceeds the Qwen2\.5\-VL reference on es\. TheModl−\-J0 A∩\\capL deltas are\+23\.87\+23\.87/\+44\.33\+44\.33/\+51\.53\+51\.53\(en/es/vi\), item\-paired bootstrap over pooled seeds \(n=6,585, 95% confidence intervals\[22\.8,25\.0\]\[22\.8,25\.0\]/\[42\.9,45\.8\]\[42\.9,45\.8\]/\[50\.3,52\.8\]\[50\.3,52\.8\], all excluding zero\)\. GeoChat\-7B \(evaluated via a weight conversion to the llava\-hf layout at its native 504 px\) is a reference, not a matched baseline: it trains on the full 318k GeoChat\-Instruct corpus, roughly20×20\\timesour budget\.†Cross\-lingual layer swap is the best of 14 training\-free mergers by es/vi A∩\\capL \(all 14 in Table[6](https://arxiv.org/html/2609.26097#A4.T6); by the en column alone TIES is strongest at 68\.5: merging preserves English, not the target language\); merge experts predate the EOS fix, so their generations are scored after repetition collapse under the disclosed\-asymmetry rule fixed in advance\. O\-LoRA shows the same failure signature as OSRM—A∩\\capL at the floor on both target languages \(Table[2](https://arxiv.org/html/2609.26097#S5.T2)\)\.
#### Benchmarks and translation\.

RS: GeoChat\-Bench scene\-classification holdout\(Kuckreja et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib18)\)\(2,195 items/language: UCMerced 2,100 \+ AID 95, main table; VRSBench\(Li et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib22)\)is excluded because its licensed imagery was unavailable, and the region and hrben/lrben subsets ship no usable references; both exclusions were preregistered\), GEOBench\-VLM multiple\-choice \(MCQ\) subset\(Danish et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib7)\)\(3,001, external validation\)\. Language retention: Belebele\(Bandarkar et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib2)\)\(900 items×\\times6 languages\) and MMLU\-ProX Lite\(Xuan et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib44)\)\(588×\\times6\)\. RS benchmarks are translated with NLLB\-200\-3\.3B\(Costa\-Jussà et al\.,[2022](https://arxiv.org/html/2609.26097#bib.bib5)\)under placeholder protection for numbers, coordinates, units, option letters, and proper names; every translated manifest passes a structural answer\-key integrity audit \(choice\-label preservation, record/ID/option alignment, reference–prompt consistency; App\.[A](https://arxiv.org/html/2609.26097#A1)\)\. Translation artifacts and their known effects\(Park et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib28)\)are why we frame this as protocol, not resource\.

#### Metrics\.

Accis*class\-identity*accuracy: the prediction names the correct class in any accepted surface form \(English canonical, translated reference, or the prompt option list’s rendering\)\. This definition is forced by two defects we quantify: scoring against the translated reference alone conflates accuracy with answer language, and our translation pipeline rendered references and prompt option lists independently, leaving 41\.1%/24\.5% of es/vi references absent from the offered options\.LF, language fidelity \(response in the query language, per language identification, LID\);Acc∩\\capLF\(correct*and*in\-language; the headline metric, written A∩\\capL in tables\); text\-languageretentionvia likelihood\-ranked MCQ accuracy on Belebele/MMLU\-ProX, which reads one forward pass and is immune to the generation\-side EOS defect described below\. Per\-system GPU\-hours are in App\.[B](https://arxiv.org/html/2609.26097#A2)\.EOS control\.Our training initially omitted the EOS terminator from supervision targets, deflating free\-form metrics of fine\-tuned models by up to7×7\\times; all main results use EOS\-corrected retraining, likelihood\-ranked metrics are immune, and the defective runs are retained as an item\-paired control \(App\.[C](https://arxiv.org/html/2609.26097#A3)\)\. Uncertainty is reported as item\-paired bootstrap intervals \(10k resamples\) for the headline comparison and seed min–max ranges elsewhere; per\-language results are always shown, never only macro averages\.

#### Systems\.

*References \(inference only\):*base LLaVA\-1\.5, official GeoChat checkpoint, Qwen2\.5\-VL\-7B\(Wang et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib41)\)\(incidental multilinguality bound\); closed models are cited, not run\.*Independent\-training mergers*\(all from the same pair of independently trained domain/language experts, D\-IND/L\-IND\): Linear, TIES\(Yadav et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib45)\), DARE\(\+\+Linear/\+\+TIES\)\(Yu et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib49)\), KnOTS\-DARE\-TIES\(Stoica et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib38)\), CAT\(Prabhakar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib33)\), and cross\-lingual layer swap\(Bandarkar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib3)\); KnOTS\-TIES was excluded when its SVD alignment exceeded a 7\-hour wall\-clock budget\.333DO\-Merging\(Zheng et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib56)\)is discussed but excluded from the executable matrix: no verified official implementation was available, and we do not benchmark against our own approximations of others’ methods\.*Merge\-aware training:*Joint\(J0\), a compute\-matchedJointat double training steps \(Joint2×\\times\),Modl\(§[3\.3](https://arxiv.org/html/2609.26097#S3.SS3)\), andFic\.*Prior orthogonal training:*OSRM\(Zhang and Zhou,[2025](https://arxiv.org/html/2609.26097#bib.bib53)\)and O\-LoRA\(Wang et al\.,[2023](https://arxiv.org/html/2609.26097#bib.bib42)\)at the same budget\. The originally plannedOrth\-W\(whitened\), module\-wise composition, SOS\-LoRA, a translate\-test pivot, D\-only/L\-only decompositions, an Aya data ablation, and the MT\-trained upper bound \(PALO recipe,Rasheed et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib34)\) were cut when the compute window contracted; each cut is disclosed here rather than silently dropped, and the associated claims are removed\.

## 5Results

Table 2:Which orthogonality?The orthogonal\-training family under one matched budget, es/vi average \(holdout Acc/LF/A∩\\capL and Belebele retention, %\)\.†Built on the pre\-fix domain expert and scored with repetition collapse \(App\.[C](https://arxiv.org/html/2609.26097#A3)\); their likelihood\-based Belebele columns are defect\-immune\.#### Which orthogonality matters\.

Table[2](https://arxiv.org/html/2609.26097#S5.T2)decomposes the recipe’s two design choices\. OSRM separates the subspaces once, at initialization, and training freely re\-entangles them; O\-LoRA constrains only the language adapter against a frozen domain adapter\. Both raise Belebele retention overJoint\(the constraint does protect text competence\) yet leave A∩\\capL at zero: fidelity is lost unless*both*updates are kept apart*while they are learned*\. The mutual, training\-long constraint is therefore not an implementation detail but the operative ingredient\.

![Refer to caption](https://arxiv.org/html/2609.26097v1/x2.png)Figure 3:All systems on one axis \(A∩\\capL, es/vi average\)\.Markers are three\-seed means at the primary hyperparameter; error bars span the seed minimum–maximum \(systems trained once have no bar;λ\\lambdaablations are in Table[3](https://arxiv.org/html/2609.26097#S5.T3)\)\. BothModlvariants sit at the Qwen2\.5\-VL reference line with seed ranges of a few points; every alternative lies at or below the untrained base\.Joint’s wide bar reflects its diverged seed’s step\-500 substitute\. The full per\-language, per\-seed breakdown is in Table[4](https://arxiv.org/html/2609.26097#A4.T4)\(App\.[D](https://arxiv.org/html/2609.26097#A4)\)\.![Refer to caption](https://arxiv.org/html/2609.26097v1/x3.png)Figure 4:Case studies across all five languages against published systems:GeoChat\-7B\(Kuckreja et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib18)\), Qwen2\.5\-VL\-7B\(Wang et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib41)\), and the cross\-lingual layer\-swap merger\(Bandarkar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib3)\)\. All outputs are verbatim generations on true satellite inputs\. Box colors: green = correct class in the query language; orange = correct class in English; red = wrong class, off\-task, or degenerate\. On vi all three published systems miss whileModlanswers correctly; on ar/hi GeoChat echoes the prompt andModlmatches Qwen’s in\-language answer verbatim with zero multilingual–multimodal data; on Swahili, the hardest language in aggregate \(Table[1](https://arxiv.org/html/2609.26097#S4.T1)\),Modlanswers cleanly in Swahili while GeoChat drifts to an English description\. The final row shows English, where every system succeeds: the contest is entirely in the target languages\. Cases illustrate individual failure modes; aggregate comparisons are in Table[1](https://arxiv.org/html/2609.26097#S4.T1)\.
#### Preregistered decision rules and outcomes\.

Figure[3](https://arxiv.org/html/2609.26097#S5.F3)places every trained system on one axis; the seed\-level spread behind each mean below can be read from it directly, and the full per\-language breakdown is in App\.[D](https://arxiv.org/html/2609.26097#A4)\. Three falsifiable predictions were frozen before the confirmatory runs \(the registration originally favored the functional hypothesis; App\.[B](https://arxiv.org/html/2609.26097#A2)\), and the data answered all three\.*\(i\) Geometry\.*Following text\-only evidence\(Zhang et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib52)\), the registration predicted that mutual orthogonality would not beatJoint\. The data show the opposite:Modlimproved A∩\\capL to0\.560\.56–0\.710\.71against≤0\.05\{\\leq\}0\.05forJointon es/vi on both stable seeds \(the diverged seed’s step\-500 substitute peaks at0\.330\.33, still≥24\{\\geq\}24points belowModl\), with paired bootstrap intervals excluding zero\.*\(ii\) Function space\.*The registration predictedFicwould beatJointand the geometric variants\.Ficimproved Belebele/MMLU retention overJointin all six cells on all three seeds, but recovered neither language fidelity nor target\-language accuracy, andModlexceeded it on every axis\.*\(iii\) Language spread\.*The registration predicted the largest gains for vi/sw\. Instead, the five\-language variant preserved en/es/vi \(A∩\\capL0\.680\.68/0\.600\.60/0\.560\.56\) and collapsed ar/hi/sw\. Per\-pair runs rule out budget dilution as the cause: trained on one language each,Modlmatches the two\-language joint deployment on es/vi \(A∩\\capL59\.259\.2/55\.755\.7\), restores full in\-language answering for ar/hi \(LF1\.01\.0vs\.0\.480\.48/0\.150\.15for per\-pairJoint\), lifts their A∩\\capL to15\.915\.9/23\.623\.6\(vs\.0\.20\.2/3\.43\.4\), and returns Belebele retention to base level, yet in\-language class naming stays far below the es/vi5656–5959\. The constraint’s repairs are universal; what remains is a script and base\-coverage boundary on naming RS classes in non\-Latin scripts, with Swahili at the floor for every system\. The registered advance criterion against the best merger could not be evaluated cleanly \(the merge experts predate the EOS fix; a clean retrain did not fit the compute window\) and is reported under the disclosed\-asymmetry rule fixed in advance \(App\.[D](https://arxiv.org/html/2609.26097#A4)\)\. One unplanned observation: under the shared no\-clipping recipe, unconstrained training diverged in 3 of 7 runs while all orthogonality\-constrained runs were stable; we report this descriptively\.

#### Ablations\.

Table[3](https://arxiv.org/html/2609.26097#S5.T3)gathers all ablations with an explicit grouping column: constraint strength, theFicconsistency weight \(no value rescues fidelity or accuracy\), the unconstrained controls, and the language\-scope question of the same recipe deployed five\-way versus per\-pair\. The originally planned wider grid \(whitened penalty, module\-wise composition, projector access, rank grid, \+Aya\) was cut under the compute window and is listed in Limitations rather than silently dropped\.

Table 3:Unified ablations \(seed 43\)\.Rows above the scope blocks are es/vi averages; scope rows are per\-language cells of the same recipe deployed five\-way \(one shared adapter\) or per\-pair \(one language per adapter\)\. Latin\-script languages survive even five\-way sharing; per\-pair training doubles ar/hi; doublingJoint’s budget raises Acc but leaves A∩\\capL at floor: compute does not substitute for the constraint\.

## 6Analysis

#### Geometry vs\. function\.

The two constraints dissociate cleanly:Modlseparates the subspaces and is the only system whose composition works, whileFicleaves geometry essentially unchanged and fails on fidelity and accuracy\. In this multimodal setting, subspace separation empirically*is*the operative mechanism, the opposite of the text\-only picture drawn byZhang et al\. \([2025](https://arxiv.org/html/2609.26097#bib.bib52)\)\. Consistent with this,Modlis also the only matched\-budget system whose text\-side likelihoods stay at the untrained base’s level, suggesting the constraint protects the base’s representations wholesale rather than trading capabilities\.

#### Interference matrix\.

Table[1](https://arxiv.org/html/2609.26097#S4.T1)doubles as a capability\-retention matrix, with English RS, target\-language RS, and target\-language text as its axes: naive training and every prior composition route sacrifice at least one axis, andModlis the only intervention positive on all three\.

#### Where do domain and language live?

The merger ranking offers indirect evidence that the text\-only picture, language identity concentrating in upper layers\(Bandarkar et al\.,[2025](https://arxiv.org/html/2609.26097#bib.bib3)\), transfers to MLLMs: the only merger with non\-trivial target\-language fidelity is the one that swaps upper layers wholesale \(layer swap, A∩\\capL 27\.2 on vi\), while every weight\-mixing merger is at the floor \(Table[6](https://arxiv.org/html/2609.26097#A4.T6)\)\. A systematic layer\-band sweep is left to future work\.

#### Case studies\.

Figure[4](https://arxiv.org/html/2609.26097#S5.F4)walks the five languages on real holdout items against the published systems: per\-pairModlanswers correctly in Spanish, Vietnamese, Arabic, and Hindi, Swahili sits at the script boundary, and the failure modes match the aggregates \(prompt echo, English drift, off\-task text\)\.

## 7Conclusion

A remote\-sensing MLLM can acquire a new language without a single multilingual RS example, but only if interference is controlled at the source\. Naive joint training of a domain and a language LoRA answers correctly in the wrong language, erases text\-only multilingual competence, and is unstable across seeds; a function\-space variant softens only the second failure, and published orthogonal\-training methods fail outright\. One symmetric mutual\-orthogonality loss term resolves all three on every seed, surpassing the untrained base and, on Spanish scene classification, the Qwen2\.5\-VL reference: in this multimodal setting, parameter geometry is the deciding factor\. The recipe applies wherever an English\-only domain MLLM and target\-language text instructions exist; code, adapters, and the preregistered protocol are released\.

## Limitations

Claims are scoped to a matched compute\-bounded budget \(1,000 steps per system, 6\.8% of the assembled corpus; the preregistered 2,500 and the plan’s larger ambitions did not fit the reserved window\); full\-convergence behavior may differ\. The domain evaluation is one benchmark family \(scene classification\); harder open\-ended RS tasks may behave differently\. The independent\-merge baselines are built from experts trained before the EOS fix and are reported under a disclosed\-asymmetry rule fixed in advance, as are OSRM and O\-LoRA, whose frozen domain expert predates the fix; SOS\-LoRA, the MT\-trained upper bound, the translate\-test pivot, and the whitened\-orthogonality and module\-wise variants were cut and not run\. The shared recipe uses no gradient clipping; the observed stability advantage of the orthogonality constraint is reported under that recipe and may shrink with clipping\. The five\-language joint variant dilutes low\-resource languages at this budget, so the recipe claim is per language pair\. Language fidelity is judged by an n\-gram LID over short class\-name answers; its error on such fragments is not separately validated\. All non\-English numbers rest on NLLB translation quality; no native\-speaker validation was performed\. Evaluation uses machine translation, not natively authored RS test sets, which do not exist beyond EN/ZH; artifacts cannot be fully excluded\(Park et al\.,[2024](https://arxiv.org/html/2609.26097#bib.bib28)\)\. The recipe presumes an English\-centric base; on multilingual backbones the language module may be partially redundant \(our Qwen2\.5\-VL reference bounds this\)\. Text\-only language LoRAs teach language form, not culturally grounded visual concepts\. We study one domain, five languages, one 7B architecture; the exploratory\-to\-confirmatory protocol and preregistered gates are reported so that negative or null confirmatory outcomes are interpretable rather than hidden\.

## Ethics Statement

Cross\-lingual RS assistants lower barriers for non\-English\-speaking communities in disaster response and land management; the same capability could ease misuse in more languages, though no imaging capability beyond the English base is added\. Machine\-translated evaluation data inherits NLLB\-200 biases\. All datasets and models used are public research releases; licenses are respected; no paid closed\-model APIs are used anywhere in the experimental pipeline\.

## References

- Ansell et al\. \(2022\)Alan Ansell, Edoardo Maria Ponti, Anna Korhonen, and Ivan Vulić\. 2022\.Composable sparse fine\-tuning for cross\-lingual transfer\.In*Proceedings of the 60th Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 1778–1796\.
- Bandarkar et al\. \(2024\)Lucas Bandarkar, Davis Liang, Benjamin Muller, Mikel Artetxe, Satya Narayan Shukla, Donald Husa, Naman Goyal, Abhinandan Krishnan, Luke Zettlemoyer, and Madian Khabsa\. 2024\.The belebele benchmark: a parallel reading comprehension dataset in 122 language variants\.In*Proceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 749–775\.
- Bandarkar et al\. \(2025\)Lucas Bandarkar, Benjamin Muller, Pritish Yuvraj, Rui Hou, Nayan Singhal, Hongjiang Lv, and Bing Liu\. 2025\.Layer swapping for zero\-shot cross\-lingual transfer in large language models\.In*International Conference on Learning Representations*, volume 2025, pages 51315–51338\.
- Chronopoulou et al\. \(2024\)Alexandra Chronopoulou, Jonas Pfeiffer, Joshua Maynez, Xinyi Wang, Sebastian Ruder, and Priyanka Agrawal\. 2024\.Language and task arithmetic with parameter\-efficient layers for zero\-shot summarization\.In*Proceedings of the Fourth Workshop on Multilingual Representation Learning \(MRL 2024\)*, pages 114–126\.
- Costa\-Jussà et al\. \(2022\)Marta R Costa\-Jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, et al\. 2022\.No language left behind: Scaling human\-centered machine translation\.*arXiv preprint arXiv:2207\.04672*\.
- Dalili and Mahdavi \(2025\)Seyed Arshan Dalili and Mehrdad Mahdavi\. 2025\.Model merging via multi\-teacher knowledge distillation\.*arXiv preprint arXiv:2512\.21288*\.
- Danish et al\. \(2025\)Muhammad Danish, Muhammad Akhtar Munir, Syed Roshaan Ali Shah, Kartik Kuckreja, Fahad Shahbaz Khan, Paolo Fraccaro, Alexandre Lacoste, and Salman Khan\. 2025\.Geobench\-vlm: Benchmarking vision\-language models for geospatial tasks\.In*Proceedings of the IEEE/CVF International Conference on Computer Vision*, pages 7132–7142\.
- Dash et al\. \(2025\)Saurabh Dash, Yiyang Nan, John Dang, Arash Ahmadian, Shivalika Singh, Madeline Smith, Bharat Venkitesh, Vlad Shmyhlo, Viraat Aryabumi, Walter Beller\-Morales, et al\. 2025\.Aya vision: Advancing the frontier of multilingual multimodality\.*arXiv preprint arXiv:2505\.08751*\.
- Du et al\. \(2025\)Yiyang Du, Xiaochen Wang, Chi Chen, Jiabo Ye, Yiru Wang, Peng Li, Ming Yan, Ji Zhang, Fei Huang, Zhifang Sui, Maosong Sun, and Yang Liu\. 2025\.[Adamms: Model merging for heterogeneous multimodal large language models with unsupervised coefficient optimization](https://doi.org/10.1109/CVPR52734.2025.00879)\.In*2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 9413–9422\.
- Geigle et al\. \(2024\)Gregor Geigle, Abhay Jain, Radu Timofte, and Goran Glavaš\. 2024\.mblip: Efficient bootstrapping of multilingual vision\-llms\.In*Proceedings of the 3rd Workshop on Advances in Language and Vision Research \(ALVR\)*, pages 7–25\.
- Geigle et al\. \(2025\)Gregor Geigle, Florian Schneider, Carolin Holtermann, Chris Biemann, Radu Timofte, Anne Lauscher, and Goran Glavaš\. 2025\.Centurio: On drivers of multilingual ability of large vision\-language model\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 2831–2881\.
- Hinck et al\. \(2024\)Musashi Hinck, Carolin Holtermann, Matthew Lyle Olson, Florian Schneider, Sungduk Yu, Anahita Bhiwandiwalla, Anne Lauscher, Shao\-Yen Tseng, and Vasudev Lal\. 2024\.Why do llava vision\-language models reply to images in english?In*Findings of the Association for Computational Linguistics: EMNLP 2024*, pages 13402–13421\.
- Hu et al\. \(2022\)Edward J Hu, Yelong Shen, Phillip Wallis, Zeyuan Allen\-Zhu, Yuanzhi Li, Shean Wang, Liang Wang, Weizhu Chen, et al\. 2022\.Lora: Low\-rank adaptation of large language models\.*ICLR*, 1\(2\):3\.
- Hu et al\. \(2024\)Jinyi Hu, Yuan Yao, Chongyi Wang, Shan Wang, Yinxu Pan, Qianyu Chen, Tianyu Yu, Hanghao Wu, Yue Zhao, Haoye Zhang, et al\. 2024\.Large multilingual models pivot zero\-shot multimodal learning across languages\.In*International conference on learning representations*, volume 2024, pages 43308–43333\.
- Hu et al\. \(2025\)Yuan Hu, Jianlong Yuan, Congcong Wen, Xiaonan Lu, Yu Liu, and Xiang Li\. 2025\.[Rsgpt: A remote sensing vision language model and benchmark](https://doi.org/10.1016/j.isprsjprs.2025.03.028)\.*ISPRS Journal of Photogrammetry and Remote Sensing*, 224:272–286\.
- Ilharco et al\. \(2023\)Gabriel Ilharco, Marco Tulio Ribeiro, Mitchell Wortsman, Suchin Gururangan, Ludwig Schmidt, Hannaneh Hajishirzi, and Ali Farhadi\. 2023\.[Editing models with task arithmetic](https://arxiv.org/abs/2212.04089)\.*Preprint*, arXiv:2212\.04089\.
- Jin et al\. \(2022\)Xisen Jin, Xiang Ren, Daniel Preotiuc\-Pietro, and Pengxiang Cheng\. 2022\.Dataless knowledge fusion by merging weights of language models\.*arXiv preprint arXiv:2212\.09849*\.
- Kuckreja et al\. \(2024\)Kartik Kuckreja, Muhammad Sohail Danish, Muzammal Naseer, Abhijit Das, Salman Khan, and Fahad Shahbaz Khan\. 2024\.Geochat: Grounded large vision\-language model for remote sensing\.In*2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 27831–27840\. IEEE\.
- Lee et al\. \(2025a\)Jung Lee, Taero Kim, and Nikhil Verma\. 2025a\.Mlm: Multi\-linguistic lora merging\.In*NeurIPS 2025 Workshop on Efficient Reasoning*\.
- Lee et al\. \(2025b\)Yeoreum Lee, Jinwook Jung, and Sungyong Baik\. 2025b\.Mitigating parameter interference in model merging via sharpness\-aware fine\-tuning\.In*International Conference on Learning Representations*, volume 2025, pages 50156–50178\.
- Li et al\. \(2023\)Haonan Li, Fajri Koto, Minghao Wu, Alham Fikri Aji, and Timothy Baldwin\. 2023\.Bactrian\-x: Multilingual replicable instruction\-following models with low\-rank adaptation\.*arXiv preprint arXiv:2305\.15011*\.
- Li et al\. \(2024\)Xiang Li, Jian Ding, and Mohamed Elhoseiny\. 2024\.Vrsbench: A versatile vision\-language benchmark dataset for remote sensing image understanding\.*Advances in Neural Information Processing Systems*, 37:3229–3242\.
- Li and Hoiem \(2017\)Zhizhong Li and Derek Hoiem\. 2017\.Learning without forgetting\.*IEEE transactions on pattern analysis and machine intelligence*, 40\(12\):2935–2947\.
- Liu et al\. \(2023\)Haotian Liu, Chunyuan Li, Qingyang Wu, and Yong Jae Lee\. 2023\.[Visual instruction tuning](https://doi.org/10.52202/075280-1516)\.In*Advances in Neural Information Processing Systems*, volume 36, pages 34892–34916\. Curran Associates, Inc\.
- Liu et al\. \(2025\)Jiaqi Liu, Lan Zhang, and Xiaoyong Yuan\. 2025\.Dyme: Dynamic multi\-concept erasure in diffusion models with bi\-level orthogonal lora adaptation\.*arXiv preprint arXiv:2509\.21433*\.
- Muhtar et al\. \(2025\)Dilxat Muhtar, Zhenshi Li, Feng Gu, Xueliang Zhang, and Pengfeng Xiao\. 2025\.Lhrs\-bot: Empowering remote sensing with vgi\-enhanced large multimodal language model\.In*Computer Vision – ECCV 2024*, pages 440–457, Cham\. Springer Nature Switzerland\.
- Ortiz\-Jimenez et al\. \(2023\)Guillermo Ortiz\-Jimenez, Alessandro Favero, and Pascal Frossard\. 2023\.Task arithmetic in the tangent space: Improved editing of pre\-trained models\.*Advances in Neural Information Processing Systems*, 36:66727–66754\.
- Park et al\. \(2024\)ChaeHun Park, Koanho Lee, Hyesu Lim, Jaeseok Kim, Junmo Park, Yu\-Jung Heo, Du\-Seong Chang, and Jaegul Choo\. 2024\.Translation deserves better: Analyzing translation artifacts in cross\-lingual visual question answering\.In*Findings of the Association for Computational Linguistics: ACL 2024*, pages 5193–5221\.
- Pfeiffer et al\. \(2022\)Jonas Pfeiffer, Gregor Geigle, Aishwarya Kamath, Jan\-Martin O Steitz, Stefan Roth, Ivan Vulić, and Iryna Gurevych\. 2022\.xgqa: Cross\-lingual visual question answering\.In*Findings of the association for computational linguistics: ACL 2022*, pages 2497–2511\.
- Pfeiffer et al\. \(2020\)Jonas Pfeiffer, Ivan Vulić, Iryna Gurevych, and Sebastian Ruder\. 2020\.Mad\-x: An adapter\-based framework for multi\-task cross\-lingual transfer\.In*Proceedings of the 2020 conference on empirical methods in natural language processing \(EMNLP\)*, pages 7654–7673\.
- Pikabea et al\. \(2025\)Iñigo Pikabea, Iñaki Lacunza, Oriol Pareras Velasco, Carlos Escolano, Aitor Gonzalez\-Agirre, Javier Hernando, and Marta Villegas\. 2025\.Breaking language barriers in visual language models via multilingual textual regularization\.In*Proceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics*, pages 299–337\.
- Po et al\. \(2024\)Ryan Po, Guandao Yang, Kfir Aberman, and Gordon Wetzstein\. 2024\.Orthogonal adaptation for modular customization of diffusion models\.In*2024 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 7964–7973\. IEEE\.
- Prabhakar et al\. \(2025\)Akshara Prabhakar, Yuanzhi Li, Karthik Narasimhan, Sham Kakade, Eran Malach, and Samy Jelassi\. 2025\.Lora soups: Merging loras for practical skill composition tasks\.In*Proceedings of the 31st International Conference on Computational Linguistics: Industry Track*, pages 644–655\.
- Rasheed et al\. \(2025\)Hanoona Rasheed, Muhammad Maaz, Abdelrahman Shaker, Salman Khan, Hisham Cholakkal, Rao M\. Anwer, Tim Baldwin, Michael Felsberg, and Fahad S\. Khan\. 2025\.Palo: A polyglot large multimodal model for 5b people\.In*Proceedings of the Winter Conference on Applications of Computer Vision \(WACV\)*, pages 1745–1754\.
- Rebelo et al\. \(2025\)Carlos Rebelo, Gil Rocha, João Daniel Silva, and Bruno Martins\. 2025\.Multilingual training\-free remote sensing image captioning\.*arXiv preprint arXiv:2512\.00887*\.
- Silva et al\. \(2024\)João Daniel Silva, João Magalhães, Devis Tuia, and Bruno Martins\. 2024\.[Multilingual vision\-language pre\-training for the remote sensing domain](https://doi.org/10.1145/3678717.3691318)\.In*Proceedings of the 32nd ACM International Conference on Advances in Geographic Information Systems*, SIGSPATIAL ’24, pages 220–232, New York, NY, USA\. Association for Computing Machinery\.
- Soni et al\. \(2025\)Sagar Soni, Akshay Dudhane, Hiyam Debary, Mustansar Fiaz, Muhammad Akhtar Munir, Muhammad Sohail Danish, Paolo Fraccaro, Campbell D Watson, Levente J Klein, Fahad Shahbaz Khan, and Salman Khan\. 2025\.[Earthdial: Turning multi\-sensory earth observations to interactive dialogues](https://doi.org/10.1109/CVPR52734.2025.01334)\.In*2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 14303–14313\.
- Stoica et al\. \(2025\)George Stoica, Pratik Ramesh, Boglarka Ecsedi, Leshem Choshen, and Judy Hoffman\. 2025\.[Model merging with svd to tie the knots](https://proceedings.iclr.cc/paper_files/paper/2025/file/0d4f8a5109c5083b5307fcd0bddae7a7-Paper-Conference.pdf)\.In*International Conference on Learning Representations*, volume 2025, pages 4501–4519\.
- Wan et al\. \(2024\)Fanqi Wan, Xinting Huang, Deng Cai, Xiaojun Quan, Wei Bi, and Shuming Shi\. 2024\.Knowledge fusion of large language models\.In*International Conference on Learning Representations*, volume 2024, pages 18303–18322\.
- Wang et al\. \(2025\)Fengxiang Wang, Hongzhen Wang, Zonghao Guo, Di Wang, Yulin Wang, Mingshuo Chen, Qiang Ma, Long Lan, Wenjing Yang, Jing Zhang, Zhiyuan Liu, and Maosong Sun\. 2025\.[Xlrs\-bench: Could your multimodal llms understand extremely large ultra\-high\-resolution remote sensing imagery?](https://doi.org/10.1109/CVPR52734.2025.01336)In*2025 IEEE/CVF Conference on Computer Vision and Pattern Recognition \(CVPR\)*, pages 14325–14336\.
- Wang et al\. \(2024\)Peng Wang, Shuai Bai, Sinan Tan, Shijie Wang, Zhihao Fan, Jinze Bai, Keqin Chen, Xuejing Liu, Jialin Wang, Wenbin Ge, et al\. 2024\.Qwen2\-vl: Enhancing vision\-language model’s perception of the world at any resolution\.*arXiv preprint arXiv:2409\.12191*\.
- Wang et al\. \(2023\)Xiao Wang, Tianze Chen, Qiming Ge, Han Xia, Rong Bao, Rui Zheng, Qi Zhang, Tao Gui, and Xuan\-Jing Huang\. 2023\.Orthogonal subspace learning for language model continual learning\.In*Findings of the Association for Computational Linguistics: EMNLP 2023*, pages 10658–10671\.
- Wang et al\. \(2026\)Zijing Wang, Mingyang Wang, Ercong Nie, Yongkang Liu, Shi Feng, Mengjie Zhao, Daling Wang, Xiaocui Yang, and Hinrich Schütze\. 2026\.DiM3: Bridging multilingual and multimodal models via direction\- and magnitude\-aware merging\.*arXiv preprint arXiv:2605\.12960*\.
- Xuan et al\. \(2025\)Weihao Xuan, Rui Yang, Heli Qi, Qingcheng Zeng, Yunze Xiao, Aosong Feng, Dairui Liu, Yun Xing, Junjue Wang, Fan Gao, et al\. 2025\.Mmlu\-prox: A multilingual benchmark for advanced large language model evaluation\.In*Proceedings of the 2025 Conference on Empirical Methods in Natural Language Processing*, pages 1513–1532\.
- Yadav et al\. \(2023\)Prateek Yadav, Derek Tam, Leshem Choshen, Colin Raffel, and Mohit Bansal\. 2023\.[Ties\-merging: Resolving interference when merging models](https://doi.org/10.52202/075280-0310)\.In*Advances in Neural Information Processing Systems*, volume 36, pages 7093–7115\. Curran Associates, Inc\.
- Yang et al\. \(2026\)Enneng Yang, Li Shen, Guibing Guo, Xingwei Wang, Xiaochun Cao, Jie Zhang, and Dacheng Tao\. 2026\.Model merging in llms, mllms, and beyond: Methods, theories, applications, and opportunities\.*ACM Computing Surveys*, 58\(8\):1–41\.
- Yang et al\. \(2024\)Enneng Yang, Zhenyi Wang, Li Shen, Shiwei Liu, Guibing Guo, Xingwei Wang, and Dacheng Tao\. 2024\.Adamerging: Adaptive model merging for multi\-task learning\.In*International Conference on Learning Representations*, volume 2024, pages 22743–22763\.
- Yang et al\. \(2025\)Jiahui Yang, Yongjia Ma, Donglin Di, Jianxun Cui, Hao Li, Wei Chen, Yan Xie, Xun Yang, and Wangmeng Zuo\. 2025\.Qr\-lora: Efficient and disentangled fine\-tuning via qr decomposition for customized generation\.In*2025 IEEE/CVF International Conference on Computer Vision \(ICCV\)*, pages 17587–17597\. IEEE\.
- Yu et al\. \(2024\)Linyan Yu, Bowen Yu, Haiyang Yu, Fei Huang, and Yongbin Li\. 2024\.Language models are super mario: Absorbing abilities from homologous models as a free lunch\.In*ICML*, volume 2, page 21\.
- Yuan et al\. \(2023\)Zhenghang Yuan, Lichao Mou, and Xiao Xiang Zhu\. 2023\.[Multilingual augmentation for robust visual question answering in remote sensing images](https://doi.org/10.1109/JURSE57346.2023.10144189)\.In*2023 Joint Urban Remote Sensing Event \(JURSE\)*, pages 1–4\.
- Yue et al\. \(2025\)Xiang Yue, Yueqi Song, Akari Asai, Seungone Kim, Jean Nyandwi, Simran Khanuja, Anjali Kantharuban, Lintang Sutawika, Sathyanarayanan Ramamoorthy, and Graham Neubig\. 2025\.Pangea: A fully open multilingual multimodal llm for 39 languages\.In*International Conference on Learning Representations*, volume 2025, pages 47758–47811\.
- Zhang et al\. \(2025\)Andi Zhang, Xuan Ding, Haofan Wang, Steven McDonagh, and Samuel Kaski\. 2025\.Rethinking inter\-lora orthogonality in adapter merging: Insights from orthogonal monte carlo dropout\.*arXiv preprint arXiv:2510\.03262*\.
- Zhang and Zhou \(2025\)Haobo Zhang and Jiayu Zhou\. 2025\.Unraveling lora interference: Orthogonal subspaces for robust model merging\.In*Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\)*, pages 26459–26472\.
- Zhang et al\. \(2024\)Wei Zhang, Miaoxin Cai, Tong Zhang, Yin Zhuang, and Xuerui Mao\. 2024\.[Earthgpt: A universal multimodal large language model for multisensor image comprehension in remote sensing domain](https://doi.org/10.1109/TGRS.2024.3409624)\.*IEEE Transactions on Geoscience and Remote Sensing*, 62:1–20\.
- Zhao et al\. \(2025\)Yiran Zhao, Wenxuan Zhang, Huiming Wang, Kenji Kawaguchi, and Lidong Bing\. 2025\.Adamergex: Cross\-lingual transfer with large language models via adaptive adapter merging\.In*Proceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\)*, pages 9785–9800\.
- Zheng et al\. \(2025\)Shenghe Zheng, Hongzhi Wang, Chenyu Huang, Xiaohui Wang, Tao Chen, Jiayuan Fan, Shuyue Hu, and Peng Ye\. 2025\.Decouple and orthogonalize: A data\-free framework for lora merging\.*arXiv preprint arXiv:2505\.15875*\.
- Zhou et al\. \(2025\)Qing Zhou, Tao Yang, Junyu Gao, Weiping Ni, Junzheng Wu, and Qi Wang\. 2025\.A benchmark for multi\-lingual vision\-language learning in remote sensing image captioning\.*arXiv preprint arXiv:2503\.04592*\.

## Appendix ATranslation Protocol

All evaluation sets are translated with a pinned local NLLB\-200\-3\.3B under placeholder protection for numbers, coordinates, units, option letters, and proper names, with SHA\-256\-checked resumable batching; no paid APIs are used anywhere\. Two pipeline defects were discovered and quantified during the confirmatory phase \(§[4](https://arxiv.org/html/2609.26097#S4)\): the class inventory was rendered independently for references and for prompt option lists, leaving 41\.1%/24\.5% of es/vi references absent from the offered options; scoring therefore uses the released class\-identity map \(option\-list\-aligned accepted surface forms, 2,195/2,195 positional alignment in all five languages\)\.

## Appendix BHyperparameters, Compute, and Preregistration

LLaVA\-1\.5\-7B, 4\-bit NF4 base, bf16 compute;D\-LoRArank 64 on LLM attention/MLP and projector,L\-LoRArank 32 on the LLM only; AdamW, lr2×10−42\{\\times\}10^\{\-4\}, no gradient clipping, micro\-batch 1, gradient accumulation 16, max length 1,024, 1,000 optimizer steps \(preregistered fallback from 2,500, executed before any run passed step 100\), checkpoints every 250, seeds 43/44/45\.Fic:λc=0\.5\\lambda\_\{c\}\{=\}0\.5primary \(0\.25/1\.00\.25/1\.0ablated\),K=64K\{=\}64,τ=1\\tau\{=\}1\.Modl:λ=0\.1\\lambda\{=\}0\.1primary \(0\.01/1\.00\.01/1\.0ablated\), factor cosine, sum reduction\. Approximate cost: J0≈\\approx7 GPU\-h/run,Fic≈\\approx14 \(detached\-teacher forwards\),Modl≈\\approx7\.3\.

#### Compute\.

All experiments ran on NVIDIA A100\-SXM4\-40 GB GPUs \(a Slurm partition of three 4\-GPU nodes\), one GPU per job; no multi\-GPU training was used\. Slurm accounting for the complete study records≈\\approx1,249 A100\-hours:≈\\approx501 for the final reported trainings \(the matched\-budget systems across seeds, plus the independent domain/language experts behind the mergers\),≈\\approx325 for evaluation,≈\\approx109 for theλ\\lambdaablations and the cut SOS\-LoRA pilot,≈\\approx115 for the superseded no\-EOS trainings retained as controls, and≈\\approx199 for failed, cancelled, or timed\-out jobs \(including merge\-construction jobs that exceeded their wall\-clock\)\.

#### Implementation\.

Python 3\.12\.13, PyTorch 2\.2\.2\+cu121, Transformers 4\.57\.3, PEFT 0\.19\.1 \(LoRA training and composition\), bitsandbytes 0\.43\.3 \(NF4 quantization\)\. Translation uses the pinned localfacebook/nllb\-200\-3\.3Bcheckpoint with greedy decoding \(do\_sample=False, no beam search, forced target\-language BOS, source truncation andmax\_new\_tokensat 512, batch size 8\)\. Language identification uses langid 1\.1\.6 constrained to the six study languages\. Evaluation\-time generation is greedy with at most 16 new tokens; likelihood metrics use a single forward pass and no sampling\.

The preregistration \(frozen gates, 22 addenda covering the environment incident, the budget fallback, the EOS defect and retraining, divergence handling, the merge\-asymmetry protocol, and this closeout\) is released with the code\. The paper was originally registered aroundFic; the reversal toModlis documented in Addenda 19–22, each written before the corresponding results existed\.

#### Input–output formats\.

*Domain stream*\(updatesD\-LoRA\): input is one RS image with an English instruction in the LLaVA chat template,<image\> USER: Classify the given image in one of the following classes\. Classes: … ASSISTANT:; the supervised target is the English answer followed by the EOS token \(tennis court</s\>\), with loss on the answer span only\.*Language stream*\(updatesL\-LoRAℓ\): input is a text\-only target\-language instruction from Bactrian\-X, no image; the target is the target\-language response plus EOS\. The two streams alternate mini\-batches; no example ever pairs a target language with an image\.*Evaluation*: holdout inputs are an RS image plus the translated classification prompt \(e\.g\.<image\> USER: Clasifique la imagen dada … ASSISTANT:\); generation is greedy with at most 16 new tokens\. Belebele/MMLU\-ProX inputs are text\-only MCQs; no generation is used there—the answer is the option letter with the highest next\-token likelihood in a single forward pass\.

## Appendix CJudging, LID, and the EOS Control

Scoring is fully deterministic: normalized exact match and token\-F1, option\-letter extraction for MCQ, likelihood\-ranked MCQ from a single forward pass, and class\-identity accuracy against the accepted\-form map\. Language identification uses langid constrained to the six study languages; answers ineligible for LID \(numbers, option letters\) are reported as coverage\. For systems trained before the EOS fix \(mergers, the frozen domain expert inside OSRM/O\-LoRA\), generations are first collapsed to their initial non\-repeating segment, a rule fixed in Addendum 21 before any such result existed\. The EOS control itself: training without a supervised terminator leaves free\-form generation running to the token limit \(3535–41%41\\%of items at 1,000 steps\), deflating token\-F1 by up to7×7\\times\(en 8\.5→\\to60\.5 for J0, same seed and data\), while single\-forward\-pass likelihood metrics are unaffected; the archived no\-EOS twin of every run permits item\-paired comparison\.

## Appendix DFull Results

Per\-seed holdout results \(Acc/LF/A∩\\capL, %\):

Table 4:Per\-seed GeoChat\-holdout results\.The diverged J0 s44 is retained as the literal record; step\-500 is its substituted baseline\.Five\-languageModlacross seeds \(A∩\\capL, %\):

Table 5:Five\-way joint training preserves en/es/vi on all three seeds and dilutes ar/hi/sw\.ar/hi answer in\-language \(LF≥99\{\\geq\}99\) but misname the class\.All fourteen mergers \(collapsed scoring; Acc/A∩\\capL per cell, %\):

Table 6:Every merger transfers capability \(en columns\) and fails language fidelity \(target\-language A∩\\capL\), mirroring naive joint training\.OSRM/O\-LoRA share this signature: A∩\\capL≤0\.05\{\\leq\}0\.05on both target languages \(Table[2](https://arxiv.org/html/2609.26097#S5.T2)\)\.![Refer to caption](https://arxiv.org/html/2609.26097v1/x4.png)Figure 5:Checkpoint curves \(seed 43, es/vi average\)\.Jointdecays monotonically on both axes as training proceeds \(A∩\\capL17\.9→0\.617\.9\\to 0\.6, Belebele53→2853\\to 28\);Ficslows but does not stop the decay;Modlis at full strength from step 250 and flat thereafter\. Interference accumulates with optimization, and the mutual constraint changes the trajectory, not just the endpoint; this also rules out under\-training as an explanation for the 1,000\-step results\.Figure[5](https://arxiv.org/html/2609.26097#A4.F5)traces all three systems across checkpoints: naive fidelity destruction accumulates with training whileModlis flat from one quarter of the budget, which is why budget\-matched comparisons are essential and why the 1,000\-step scope is not an under\-training artifact\.

## Appendix ECase Studies

Figure[4](https://arxiv.org/html/2609.26097#S5.F4)shows representative items with the true satellite inputs\.