说导航器的语言：面向冻结空中VLN智能体的轨迹接地指令翻译

# Speaking the Navigator’s Language: Trajectory-Grounded Instruction Translation for Frozen Aerial VLN Agents
Source: [https://arxiv.org/html/2610.10635](https://arxiv.org/html/2610.10635)
Zhe LiuXiaogang XuJiafei WuChunyi ZhouYuan SuRui ZengTianyu DuKelu YaoChao LiShouling Ji

###### Abstract

Aerial vision\-and\-language navigation \(VLN\) agents are typically trained on detail\-rich, trajectory\-aligned commands, whereas users issue short, intent\-driven instructions; on a frozen OpenFly navigator, this*instruction gap*drops success rate \(SR\) from31\.03%31\.03\\%to11\.33%11\.33\\%\. To scale translator training, we prompt a language model with human\-written style examples to convert original commands into paired, intent\-centered Weak commands, which yield15\.27%15\.27\\%SR\. We introduce theTrajectory\-Grounded Instruction Translator \(TGIT\), a front\-end that keeps the navigator frozen and translates Weak inputs into agent\-executable commands by learning from its trajectory outcomes\. The resulting Weak\-trained translator raises Weak\-input SR to37\.93%37\.93\\%and transfers zero\-shot to real human instructions \(11\.33%→32\.51%11\.33\\%\{\\rightarrow\}32\.51\\%\); it also improves held\-out OpenFly \(4\.95%→20\.79%4\.95\\%\{\\rightarrow\}20\.79\\%\) and yields recovery on CityNav and AirVLN\.

1Zhejiang University

2Zhejiang Lab

\{chan\_xi, zhe\.liu\}@zju\.edu\.cn, xiaogangxu00@gmail\.com,

\{jiafeiwu, zhouchunyi, yuansu, ruizeng24, zjradty\}@zju\.edu\.cn,

\{yaokelu, lichao\}@zhejianglab\.org, sji@zju\.edu\.cn

## 1Introduction

Vision\-and\-Language Navigation \(VLN\) has progressed from indoor instruction following\([Anderson et al\. 2018](https://arxiv.org/html/2610.10635#bib.bib33);[Hao et al\. 2020](https://arxiv.org/html/2610.10635#bib.bib15);[Qiao et al\. 2022](https://arxiv.org/html/2610.10635#bib.bib18);[Zhang et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib10)\)to aerial platforms, where UAV agents must act under continuous 3D motion, large viewpoint changes, and fine\-grained spatial constraints\([Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29);[Yao et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib26);[Wang et al\. 2025b](https://arxiv.org/html/2610.10635#bib.bib28);[Saxena et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib7);[Sautenkov et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib1)\)\. Yet aerial VLN agents are commonly trained on detail\-dense, trajectory\-aligned instructions produced by templates or vision–language generators\([Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32);[Wang et al\. 2023b](https://arxiv.org/html/2610.10635#bib.bib9);[Fan et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib30)\), whereas users tend to issue shorter, intent\-driven commands\.

We begin with a simple controlled probe: keeping the routes, initial conditions, and UAV navigator fixed, we change only the instruction source\. If the navigator understood its training and deployment interfaces equally well, its performance should remain stable\. Instead, using natural instructions collected from 16 annotators for 203 OpenFly episodes, SR drops from31\.03%31\.03\\%with dataset instructions to11\.33%11\.33\\%with Human instructions \(Figure[1](https://arxiv.org/html/2610.10635#S1.F1)\)\. This behavioral collapse reveals an*instruction gap*: deployment language fails to activate capabilities available through the same frozen agent’s training\-language interface\.

Example: Instruction Gap in UAV Navigation\.Dataset instruction: “Turn right and proceed straight for 30 meters until reaching the gray rectangular building with a flat roof\.”Human\-style instruction: “Fly toward the building ahead\.”

Figure 1:Instruction gap and recoverable activation on a frozen OpenFly\-Agent\.\(a\) Median instruction length collapses from Orig to Weak to Human\. \(b\) On the 203\-episode evaluation set \(Selected\-Seen\), SR drops from31\.03%31\.03\\%\(Orig\) to15\.27%15\.27\\%\(Weak\) and11\.33%11\.33\\%\(Human\)\. TGIT\(W\) recovers to37\.93%37\.93\\%on Weak; the same adapter transfers zero\-shot as TGIT\(H\) to32\.51%32\.51\\%on Human\.Closing this gap is hard for a frozen navigator trained on trajectory\-aligned Orig commands: short Human instructions must be rewritten into language that matches its learned associations, and two equally plausible expansions can still induce different actions\. Agent\-centric retraining methods can absorb this mismatch\([Kamath et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib8);[Wang et al\. 2020](https://arxiv.org/html/2610.10635#bib.bib21)\), but are costly and may change perception and control, making it hard to tell whether gains come from adapting to new language or from altering the navigator itself\([Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32);[Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29);[Wang et al\. 2025b](https://arxiv.org/html/2610.10635#bib.bib28)\)\. Instruction\-centric rewriting methods instead map weak inputs back to an Orig sentence\([Shah et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib2);[Zhou et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib12);[Zhang and Kordjamshidi 2023](https://arxiv.org/html/2610.10635#bib.bib35)\), yet matching that text does not tell whether a particular navigator will succeed\. We therefore keep the navigator fixed and learn agent\-executable translations from its trajectory outcomes, not from text similarity alone\.

This observation also determines which failures should supervise the translator\. Training on arbitrary failures would conflate an instruction\-interface mismatch with navigation skills that the backbone does not possess\. We therefore focus on paired cases where the original instruction succeeds, but its weakened counterpart fails under the same route, initial condition, simulator, and policy\. Orig success serves as a capability witness, while Weak failure isolates a recoverable interface intervention\. Human annotation is too costly to scale for translator training, so we construct Weak instead: using a few Human\-written instructions as style examples, an LM converts each Orig command into a shorter, intent\-centered counterpart\.

Building on these choices, we introduceTGIT\(Trajectory\-GroundedInstructionTranslator\)\. It freezes the UAV policy and adapts only a LoRA VLM front\-end that translates Weak inputs into agent\-executable commands\. Starting from Orig instructions in the target agent’s training dataset and their LM\-generated Weak counterparts, TGIT selects hard cases, proposes multiple candidates, executes them with the frozen navigator, and converts weighted trajectory outcomes into preferences for multi\-round DPO with pool resampling\. The resulting translator is then evaluated on Weak and transferred zero\-shot to Human, testing whether execution\-grounded preferences activate the agent without imitating a single Orig sentence\.

On the 203\-episode OpenFly evaluation, TGIT raises SR from15\.27%15\.27\\%to37\.93%37\.93\\%\(2\.48×2\.48\{\\times\}\), exceeding Orig \(31\.03%31\.03\\%\), ICL, and SFT; the same Weak\-trained translator raises Human\-input SR zero\-shot from11\.33%11\.33\\%to32\.51%32\.51\\%\(2\.87×2\.87\{\\times\}\)\. Held\-out OpenFly scenes and two additional frozen agents show the same relative activation, but their absolute success remains bounded by the navigator itself\. These results support a bounded claim: TGIT repairs recoverable instruction\-interface failures, while strengthening the frozen navigator remains an orthogonal agent\-centric problem\.

Our contributions are threefold:

- •Recoverable instruction gap\.We provide Human evidence for the mismatch between deployment and training language, and operationally identify recoverable failures through original\-success / Human\-failure pairs\.
- •Agent\-executable instruction translation\.We introduce TGIT, a plug\-and\-play front\-end that can be applied to any frozen VLN agent using only that agent’s own training instructions and their Weak counterparts: it builds trajectory\-grounded preference data from hard cases and multi\-round DPO to translate underspecified inputs into executable commands\.
- •Controlled cross\-agent validation\.TGIT outperforms Weak, ICL, and SFT on OpenFly, transfers zero\-shot from Weak training to Human inputs, and yields consistent relative gains across held\-out environments and multiple frozen UAV agents\.

The appendix complements the main results with preliminary evidence, validity checks, and reproducibility details\. A preliminary study further characterizes training\-instruction style and agent sensitivity to underspecification, strengthening the instruction\-gap motivation\. Paired lexical and embedding analyses further show that Weak moves from Orig toward Human\. Complete prompts, dataset statistics, optimization settings, translator\-scale diagnostics, and same\-route trajectory comparisons support reproduction and expose how the learned translations change agent behavior\.

## 2Related Work

Prior VLN and embodied\-language research largely falls into two axes: improving the navigator, and rewriting or generating language for clarity and coverage\([Zhang et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib10);[Lin et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib5);[Ma et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib4)\)\. TGIT is not another point on either axis\. It keeps the aerial policy frozen and learns an instruction interface that activates competence already present in that policy\.

#### Agent\-centric aerial VLN\.

VLN was established in photorealistic indoor environments\([Anderson et al\. 2018](https://arxiv.org/html/2610.10635#bib.bib33)\)and later extended toward continuous control, long\-horizon navigation, and sim\-to\-real evaluation\([Anderson et al\. 2021](https://arxiv.org/html/2610.10635#bib.bib14);[Song et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib13);[Lin et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib16)\)\. AerialVLN introduced continuous outdoor UAV flight, while CityNav, OpenFly, and AeroVerse scale aerial environments, trajectories, and embodied controllers\([Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29);[Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32);[Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Yao et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib26)\)\. These advances improve navigation capability, typically under detailed, trajectory\-aligned training instructions\.

#### Instruction\-centric language interfaces\.

Synthetic instruction generation scales VLN supervision\([Huang et al\. 2022](https://arxiv.org/html/2610.10635#bib.bib11);[Kamath et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib8);[Wang et al\. 2023b](https://arxiv.org/html/2610.10635#bib.bib9);[Gopinathan et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib20)\), while landmark\-centered generation and ULN expose the importance of specificity\([Wang et al\. 2022](https://arxiv.org/html/2610.10635#bib.bib17);[Feng et al\. 2022](https://arxiv.org/html/2610.10635#bib.bib34)\)\. Closest to our setting, VLN\-Trans decomposes instructions into landmark\-discriminative sub\-instructions\([Zhang and Kordjamshidi 2023](https://arxiv.org/html/2610.10635#bib.bib35)\); T\-araVLN translates language for agricultural navigation\([Zhao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib36)\); and RAM rewrites observation–instruction pairs for augmentation and unseen generalization\([Wei et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib37)\)\. These methods optimize clarity, recognizability, or training\-data coverage\([Wang et al\. 2023a](https://arxiv.org/html/2610.10635#bib.bib31)\)\.

#### Our position\.

TGIT never updates the UAV navigator, and it ranks candidate translations with simulator rollouts and updates only a LoRA front\-end via preference learning\([Rafailov et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib24);[An et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib23);[Calandriello et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib22)\)\. Multimodal preference methods often align VLM responses or retrain the VLA itself\([Xing et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib38);[Hung et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib39)\); TGIT instead uses original\-success / weak\-failure hard cases to restrict supervision to episodes where the navigator has already demonstrated the required skill\. Thus TGIT targets interface activation rather than policy retraining, instruction imitation, or data augmentation\.

![Refer to caption](https://arxiv.org/html/2610.10635v1/overview.png)Figure 2:The overview of TGIT\.\(a\)*Hard\-case pool*: keep episodes withS⁡\(π,Im\)=1S\(\\pi,I\_\{m\}\)\{=\}1andS⁡\(π,Iw\)=0S\(\\pi,I\_\{w\}\)\{=\}0so that capability exists but the weak interface fails\. \(b\)*Candidate generation*: VLM translatorfθf\_\{\\theta\}takes\(Iw,𝒪1:M\)\(I\_\{w\},\\mathcal\{O\}\_\{1:M\}\)and emitsN=3N\{=\}3executable rewrites; only LoRAAA/BBon the LM decoder is trainable\. \(c\)*Reward\+\+multi\-round LoRA–DPO*: frozen rollouts score eachyiy\_\{i\}byR=0\.5​SR\+0\.3​SPL\+0\.2​e−0\.1​dnormR\{=\}0\.5\\,\\mathrm\{SR\}\{\+\}0\.3\\,\\mathrm\{SPL\}\{\+\}0\.2\\,e^\{\-0\.1d\_\{\\mathrm\{norm\}\}\}\(SR: success rate; SPL: path efficiency;dnormd\_\{\\mathrm\{norm\}\}: normalized final distance; blue flags: targets\), form preferencesy\+≻y−y^\{\+\}\\\!\\succ\\\!y^\{\-\}, updateAA/BB, then chain the adapter and resample hard cases for the next round\. \(d\)*Training–deployment contract*: evaluation collapses to one forward pass—no reward, preference, or policy update \(Eq\.[2](https://arxiv.org/html/2610.10635#S3.E2)\)\.

## 3Methodology

### 3\.1Problem Formulation

#### System actors and freeze boundary\.

Our system contains three actors: an aerial environmentℰ\\mathcal\{E\}; a pretrained UAV navigatorπ\\pi; and a front\-end instruction translatorfθf\_\{\\theta\}\. The navigator consumes an instructionIIand executes a trajectory𝒯^=π⁡\(I,ℰ\)\.\\hat\{\\mathcal\{T\}\}=\\pi\(I,\\mathcal\{E\}\)\.Throughout adaptation and evaluation,π\\piand its controller are frozen;θ\\thetais the only trainable parameter set\. The translator also observes visual context𝒪1:M=\{om\}m=1M\\mathcal\{O\}\_\{1:M\}=\\\{o\_\{m\}\\\}\_\{m=1\}^\{M\}, whereM=1M\{=\}1is a single RGB frame andM\>1M\{\>\}1provides multi\-frame context\. This follows aerial\-VLN annotation protocols in which language is grounded in rendered observations\([Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32);[Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29)\)\. Visual context supplies scene evidence to the front\-end VLM, but no rewards, actions, goal labels, or policy updates are available to it at deployment\.

#### Instruction interface and objective\.

Conventional aerial VLN trainsπ\\piwith detail\-rich, trajectory\-aligned instructionsIm∼𝒟mI\_\{m\}\\sim\\mathcal\{D\}\_\{m\}\([Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32);[Wang et al\. 2023b](https://arxiv.org/html/2610.10635#bib.bib9)\), whereas deployment instructions may follow a Human distribution𝒟h≠𝒟m\\mathcal\{D\}\_\{h\}\\neq\\mathcal\{D\}\_\{m\}\. For controlled optimization, we use Human\-informed LM\-weakened inputsIw∼𝒟wI\_\{w\}\\sim\\mathcal\{D\}\_\{w\};𝒟w\\mathcal\{D\}\_\{w\}is a paired stress distribution, not an estimator of𝒟h\\mathcal\{D\}\_\{h\}\. GivenIwI\_\{w\}and𝒪1:M\\mathcal\{O\}\_\{1:M\}, the translator emitsI~=fθ\(Iw,𝒪1:M\),\\tilde\{I\}=f\_\{\\theta\}\(I\_\{w\},\\mathcal\{O\}\_\{1:M\}\),which the unchanged navigator executes as

𝒯^=π⁡\(I~,ℰ\)\.\\hat\{\\mathcal\{T\}\}=\\pi\(\\tilde\{I\},\\mathcal\{E\}\)\.\(1\)Let𝒯=\(s1,…,sT\)\\mathcal\{T\}=\(s\_\{1\},\\ldots,s\_\{T\}\)denote the reference UAV trajectory, with each state containing 3D position, orientation, and altitude\. We optimize only the interface \(subject toπ\\pifixed\):

maxθ\\displaystyle\\max\_\{\\theta\}𝔼\(Iw,𝒯,𝒪1:M\)∼𝒟w\\displaystyle\\mathbb\{E\}\_\{\(I\_\{w\},\\mathcal\{T\},\\mathcal\{O\}\_\{1:M\}\)\\sim\\mathcal\{D\}\_\{w\}\}\(2\)\[R\(π\(fθ\(Iw,𝒪1:M\),ℰ\),𝒯\)\],\\displaystyle\\Big\[R\\big\(\\pi\(f\_\{\\theta\}\(I\_\{w\},\\mathcal\{O\}\_\{1:M\}\),\\mathcal\{E\}\),\\mathcal\{T\}\\big\)\\Big\],whereR⁡\(⋅\)R\(\\cdot\)measures trajectory\-level execution quality\.

#### Design goals\.

Equation[2](https://arxiv.org/html/2610.10635#S3.E2)induces three requirements aligned with the challenges in the Introduction\. First,*capability identification*: supervision should target episodes whose required navigation skill has already been demonstrated, rather than asking translation to repair missing perception or control\. Second,*frozen\-interface adaptation*: the method should learn an agent\-specific executable language while preserving heterogeneous UAV policies\. Third,*execution\-grounded supervision*: learning must use expensive, non\-differentiable rollout outcomes and accommodate multiple valid command phrasings\. For these, TGIT addresses these requirements through hard\-case selection, a parameter\-efficient front\-end, and trajectory\-ranked preference optimization, respectively\.

### 3\.2Trajectory\-Grounded Instruction Translation

#### Overview: closed training–deployment loop\.

Figure[2](https://arxiv.org/html/2610.10635#S2.F2)summarizes TGIT\. Training first selects original\-success / weak\-failure hard cases\. For each sampled weak input,fθf\_\{\\theta\}generatesNNcandidate commands; the frozen navigator executes every candidate; a weighted trajectory reward converts their outcomes into pairwise preferences; and DPO updates only the LoRA translator before the hard\-case pool is resampled for the next round\. At deployment, this loop collapses to one forward pass: a deployment\-style instruction and available RGB frames enterfθf\_\{\\theta\}, one executable command is emitted, andπ\\piflies unchanged\. Simulator rewards, preference pairs, and reference trajectories are training\-only signals\. Section[4\.3](https://arxiv.org/html/2610.10635#S4.SS3)separately tests pool construction, reward ranking, and translator backbone choices without redefining the primary protocol\.

#### Hard\-case pool construction\.

Under a limited execution budget, random sampling yields many uninformative episodes: either the navigator already fails under Orig, or Weak happens to succeed by chance\. We therefore construct a compact hard\-case poolDpoolD\_\{\\mathrm\{pool\}\}\. Each instance is selected such that the original dataset instruction succeeds, while the corresponding weakened human\-style instruction fails under the same frozen agent\. WritingS⁡\(π,I,ℰ\)∈\{0,1\}S\(\\pi,I,\\mathcal\{E\}\)\\in\\\{0,1\\\}for episode success, the recoverable candidate set is

Dpool=\{\(Im,Iw,ℰ\):\\displaystyle D\_\{\\mathrm\{pool\}\}=\\\{\(I\_\{m\},I\_\{w\},\\mathcal\{E\}\):S⁡\(π,Im,ℰ\)=1,\\displaystyle S\(\\pi,I\_\{m\},\\mathcal\{E\}\)=1,\(3\)S\(π,Iw,ℰ\)=0\}\.\\displaystyle S\(\\pi,I\_\{w\},\\mathcal\{E\}\)=0\\\}\.These original\-success / weak\-failure cases isolate the instruction interface mismatch: the agent has demonstrated the required navigation ability, but the weakened command is not executable enough\. In practice, we build a pool of100100hard cases for each UAV agent and sampleK=20K\{=\}20cases per round, resampling across rounds to expose diverse failure modes\. Failures under bothImI\_\{m\}andIwI\_\{w\}are not used as evidence of a recoverable language problem\. Each pair holds the route, initial condition, simulator, and policy fixed while changing only the instruction: Orig success witnesses capability, whereas Weak failure isolates the interface intervention\. Section[4\.3](https://arxiv.org/html/2610.10635#S4.SS3)ablates this selection and resampling against random and fixed\-subset alternatives\.

#### Candidate generation and execution\.

For each sampled instructionx∈Dpoolx\\in D\_\{\\mathrm\{pool\}\}, the translator generatesN=3N=3candidate executable commands\{yi\}i=1N\\\{y\_\{i\}\\\}\_\{i=1\}^\{N\}conditioned on\(x,𝒪1:M\)\(x,\\mathcal\{O\}\_\{1:M\}\)under format constraints\. The translator conditions on visual context𝒪1:M\\mathcal\{O\}\_\{1:M\}as an input prior; Section[4\.3](https://arxiv.org/html/2610.10635#S4.SS3)varies frame count \(e\.g\., one vs\. four frames\) to test this prior\. We use complementary decoding settings: one deterministic setting for stable command structure and stochastic settings for semantic variation\. During training, the original dataset instruction is also included as a strong execution reference, but inference uses only the human\-style input and available observations\. Each candidateyiy\_\{i\}is executed by the frozen UAV agent, producing a trajectoryτi\\tau\_\{i\}with success, path efficiency, and final\-distance measurements\. Execution is essential because text similarity cannot identify an executable command: close paraphrases may induce different behavior, while distinct surface forms may produce equally successful trajectories\([Daniele et al\. 2017](https://arxiv.org/html/2610.10635#bib.bib19);[Liang et al\. 2022](https://arxiv.org/html/2610.10635#bib.bib6);[Zhou et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib12)\)\.

#### Weighted trajectory reward and preferences\.

The primary protocol ranks candidates using

R⁡\(τi,𝒯\)=α​SRi\+β​SPLi\+γ​e−λ​dinorm,R\(\\tau\_\{i\},\\mathcal\{T\}\)=\\alpha\\,\\mathrm\{SR\}\_\{i\}\+\\beta\\,\\mathrm\{SPL\}\_\{i\}\+\\gamma\\,e^\{\-\\lambda d\_\{i\}^\{\\mathrm\{norm\}\}\},\(4\)whereSRi\\mathrm\{SR\}\_\{i\},SPLi\\mathrm\{SPL\}\_\{i\}, andNEi\\mathrm\{NE\}\_\{i\}follow standard VLN metrics\([Anderson et al\. 2018](https://arxiv.org/html/2610.10635#bib.bib33)\),SPLi=ℓi/max⁡\(pi,ℓi\)\\mathrm\{SPL\}\_\{i\}=\\ell\_\{i\}/\\max\(p\_\{i\},\\ell\_\{i\}\)is path efficiency \(zero for failures\), anddinorm=NEi/max⁡\(ℓi,ϵ\)d\_\{i\}^\{\\mathrm\{norm\}\}=\\mathrm\{NE\}\_\{i\}/\\max\(\\ell\_\{i\},\\epsilon\)normalizes final navigation error by reference\-path length\. At evaluation time, reported SPL is the mean of these efficiencies over successful episodes only\. We set\(α,β,γ\)=\(0\.5,0\.3,0\.2\)\(\\alpha,\\beta,\\gamma\)=\(0\.5,0\.3,0\.2\)andλ=0\.1\\lambda=0\.1, making task success dominant while retaining dense efficiency and distance signals for failed or near\-tied candidates\. The reward\-ranking ablation in Section[4\.3](https://arxiv.org/html/2610.10635#S4.SS3)compares Success\-only and lexicographic alternatives, showing that recovery is not tied to these exact weights\. We convert trajectory rewards into pairwise preferences:

yi≻yjifR⁡\(τi,𝒯\)\>R⁡\(τj,𝒯\)\.y\_\{i\}\\succ y\_\{j\}\\quad\\text\{if\}\\quad R\(\\tau\_\{i\},\\mathcal\{T\}\)\>R\(\\tau\_\{j\},\\mathcal\{T\}\)\.\(5\)This transforms non\-differentiable rollout outcomes into a stable supervision signal for preference learning\. Unlike pure SFT on a single target phrasing, preference pairs teach the translator which of several plausible commands actually activateπ\\pi\. Near\-tied outcomes are discarded to avoid supervision from simulator noise\. The weighted scalar above remains the primary TGIT reward\.

#### Multi\-round DPO optimization\.

We optimize the translator with Direct Preference Optimization \(DPO\)\([Rafailov et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib24)\), using LoRA\([Hu et al\. 2021](https://arxiv.org/html/2610.10635#bib.bib25)\)to update only a small set of VLM parameters\. Each round starts from and references the round\-matched SFT adapter, samplesK=20K=20instructions fromDpoolD\_\{\\mathrm\{pool\}\}, generates and executes candidates, constructs preference pairs with a minimum reward gap, and updatesfθf\_\{\\theta\}\. Adapters are chained across rounds so later rounds refine an already grounded interface rather than restarting from the base VLM\. As the translator improves, its candidates produce better trajectories, which in turn provide more reliable preferences for the next round—closing the feedback loop\. Throughout this process, the UAV policyπ\\piis never modified; all gains come from aligning human\-style inputs with the frozen agent’s executable command space\. Each round changes the sampled hard cases and candidates, but not the navigator or evaluation protocol\. Resampling broadens failure coverage, while adapter chaining preserves executable patterns\. Unlike general online preference collection\([Calandriello et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib22)\), TGIT evaluates language actions with a fixed policy\.

#### Training–deployment contract\.

Adaptation may access simulator rollouts, reference paths, and trajectory metrics only to construct preference pairs\. Evaluation removes all three: the translator receives the deployment instruction and available RGB context, emits one command, and then exits the control loop\. The frozen navigator consumes that command using its original controller and stopping rule\. This contract prevents reward or trajectory leakage at deployment and attributes behavioral changes to the learned interface rather than to policy updates\.

## 4Experiments

### 4\.1Experimental Setup

We evaluate TGIT on three frozen UAV navigation agents: OpenFly\-Agent\([Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27)\), CityNav\-Agent\([Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32)\), and AirVLN\-Agent\([Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29)\)\. All navigation policies remain unchanged; only the front\-end translator is trained\. The translator is Qwen2\.5\-VL\-7B\-Instruct with LoRA\([Hu et al\. 2021](https://arxiv.org/html/2610.10635#bib.bib25)\)\(r=4r\{=\}4,α=8\\alpha\{=\}8\) on language modules\. For each agent, we build a hard\-case pool of100100original\-success / weak\-failure instructions, sampleK=20K\{=\}20cases per round for55rounds, and generateN=3N\{=\}3candidates with visual frames\. DPO starts from and references the round\-matched SFT adapter, usesβ=0\.1\\beta\{=\}0\.1, a learning rate of2×10−52\{\\times\}10^\{\-5\}, and discards pairs with a reward gap below0\.050\.05\. Adapters carry across rounds; the navigator remains fixed\.

These settings define the main protocol; Section[4\.3](https://arxiv.org/html/2610.10635#S4.SS3)ablates hard\-case pool construction, reward ranking, visual frames, and translator backbone as diagnostics\.

#### Splits and evaluation\.

We factor evaluation scenes by two axes used in prior aerial VLN protocol: whether the*translator*was optimized on that scene \(Selectedvs\.Unselected\), and whether the frozen*agent*trained on that scene \(Seenvs\.Unseen\)\([Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Lee et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib32);[Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29)\)\. Selected\-Seen / Unselected\-Seen / Unseen thus contain203/202/404203/202/404OpenFly,326/180/220326/180/220CityNav, and120/192/147120/192/147AirVLN episodes\. Selected\-Seen is the pool\-construction scene for TGIT; Unselected\-Seen is agent\-seen but translator\-unselected; Unseen is held out for the agent\. Each closed\-loop rollout lasts at most100100agent steps, and success requires final distance below2020m\.

#### Baselines and Human transfer\.

We compare Orig, Weak, ICL, and SFT under the Weak protocol; TGIT\(W\) is the primary result\. On OpenFly Selected\-Seen, sixteen annotators provide one natural instruction per episode\. Human anchors the deployment gap; Weak is the scalable, Human\-informed stress protocol\. We also report zero\-shot TGIT\(H\), which only reuses the TGIT\(W\) adapter\.

#### Protocol consistency\.

Within each agent and split, methods use the same episode set, initial conditions, rollout horizon, success threshold, controller, and metric implementation\. Only the instruction path and translator update differ\. Table[1](https://arxiv.org/html/2610.10635#S4.T1)reports the complete matched protocol; reward and translator\-backbone ablations are explicitly diagnostic because they reuse narrower candidate or frame settings\.

Controlled comparison\.All methods keep the navigator frozen; Human is gap evidence rather than an adaptation baseline\.

We report Success Rate \(SR\), success\-conditional path efficiency \(SPL\), and Navigation Error \(NE\)\([Anderson et al\. 2018](https://arxiv.org/html/2610.10635#bib.bib33);[Gao et al\. 2025](https://arxiv.org/html/2610.10635#bib.bib27);[Liu et al\. 2023](https://arxiv.org/html/2610.10635#bib.bib29)\)\. SR is the fraction of episodes that reach the goal\. SPL is the mean ofℓ/max⁡\(p,ℓ\)\\ell/\\max\(p,\\ell\)over successful episodes only; failed episodes are excluded\. Thus SPL measures how efficiently successful trajectories reach the goal and is not upper\-bounded by SR\. Higher SR/SPL and lower NE are better\.

Table 1:Main navigation results on three frozen UAV agents\. We report Success Rate \(SR, %\), success\-conditional path efficiency \(SPL, %\), and Navigation Error \(NE, meters\)\. SPL is averaged only over successful episodes\. TGIT\(H\) and TGIT\(W\) denote Human and Weak evaluation inputs, respectively\.HumanandTGIT\(H\)are evaluated only on OpenFly Selected\-Seen \(203 episodes\) due to the high cost of human annotation; other Human\-input cells are “–”\. TGIT\(H\) reuses the Weak\-trained TGIT\(W\) adapter zero\-shot, without Human\-specific training\. TGIT\(W\) deltas are against Orig \(negative NE is better\)\.

### 4\.2Results

#### The instruction gap is behavioral\.

On the same203203OpenFly Selected\-Seen episodes and frozen navigator, SR drops from31\.03%31\.03\\%with Orig to15\.27%15\.27\\%with Weak and11\.33%11\.33\\%with Human\. Because the route, initial condition, controller, and evaluation remain fixed, this collapse isolates instruction source as the changed interface variable\. The lower Human result also shows that the gap is not merely an artifact of LM weakening\. Without Human\-specific training, the same Weak\-trained TGIT adapter raises Human\-input SR from11\.33%11\.33\\%to32\.51%32\.51\\%\(2\.87×2\.87\{\\times\}\), confirming that interface activation transfers beyond the Weak proxy\. Thus the deployment mismatch is behavioral, not only a difference in wording or length\.

#### Trajectory grounding recovers the interface\.

Direct Weak execution reaches15\.27%15\.27\\%SR; prompt\-only ICL improves modestly to19\.70%19\.70\\%, and SFT target imitation recovers to29\.55%29\.55\\%but remains below Orig\. TGIT reaches37\.93%37\.93\\%, gaining22\.6622\.66points over Weak,8\.388\.38over SFT, and6\.906\.90over Orig\. The controls identify the missing supervision: ICL rewrites without agent feedback, while SFT learns one executable target without comparing alternative commands under rollout outcomes\. TGIT instead executes and ranks candidates before updating only the translator\. This pattern is consistent with trajectory\-grounded preference learning supplying information beyond prompting and target imitation; Section[4\.3](https://arxiv.org/html/2610.10635#S4.SS3)separately tests the hard\-case and preference\-design mechanisms\.

#### Transfer reveals a backbone\-bounded regime\.

Beyond the pool\-construction environment, TGIT raises OpenFly SR from14\.11%14\.11\\%to25\.25%25\.25\\%on Unselected\-Seen and from4\.95%4\.95\\%to20\.79%20\.79\\%on Unseen\. The relative activation also transfers across frozen agents: CityNav improves on Selected\-Seen \(6\.11%→10\.00%6\.11\\%\\rightarrow 10\.00\\%\), Unselected\-Seen \(1\.84%→9\.20%1\.84\\%\\rightarrow 9\.20\\%\), and Unseen \(0\.00%→3\.64%0\.00\\%\\rightarrow 3\.64\\%\); AirVLN improves on Selected\-Seen \(0\.83%→7\.50%0\.83\\%\\rightarrow 7\.50\\%\) and Unselected\-Seen \(0\.00%→1\.56%0\.00\\%\\rightarrow 1\.56\\%\)\. The OpenFly gains outside its100100\-case pool are inconsistent with a pool\-memorization\-only explanation, while the cross\-agent gains show that the activation pattern is not specific to one navigator\([Wang et al\. 2025a](https://arxiv.org/html/2610.10635#bib.bib3);[Anderson et al\. 2021](https://arxiv.org/html/2610.10635#bib.bib14)\)\. TGIT primarily expands success coverage rather than jointly optimizing every navigation metric\. Its OpenFly Selected\-Seen SPL \(24\.85%24\.85\\%\) remains below Orig and SFT, while the high Human SPL describes only the11\.33%11\.33\\%successful subset\. At the agent level, CityNav and AirVLN exhibit relative recovery but low absolute SR, and every AirVLN method remains at0\.00%0\.00\\%on Unseen\. This dual pattern—transferable relative activation but backbone\-bounded absolute success—defines the claim: TGIT repairs recoverable interface failures but does not supply missing navigation skill\.

### 4\.3Ablation Study

Table 2:Multi\-round ablation on data pool construction \(OpenFly\-Agent\)\. Results areSR \(%\) / SPL \(%\) / NE \(m\)\.Selected: original succeeds and weakened fails\.Pool: maintain 100 candidates and sample 20 per round\.Figure 3:Hard cases, failure stage, and visual context\.\(a\) Pool curves \(Table[2](https://arxiv.org/html/2610.10635#S4.T2)\); Selected\-Pool Round 5 matches Table[1](https://arxiv.org/html/2610.10635#S4.T1)\. \(b\) Failure stages with success fractions matched to Table[1](https://arxiv.org/html/2610.10635#S4.T1); TGIT early/late proportions use the nearest Selected\-Pool diagnostic after renormalizing success to77/20377/203\. \(c\) f1∗/f4∗diagnostics; “full” is TGIT in Table[1](https://arxiv.org/html/2610.10635#S4.T1)\.We ablate TGIT on OpenFly\-Agent \(*Airsim\_18*\), which shows clear sensitivity to instruction style in Table[1](https://arxiv.org/html/2610.10635#S4.T1)\. The ablations isolate three factors behind that primary TGIT result: which episodes are supervised, how rollouts become preferences, and which translator backbone is used\.

#### Hard\-case selection and pool resampling\.

Table[2](https://arxiv.org/html/2610.10635#S4.T2)compares random vs\. hard\-case selection and full\-subset vs\. pool resampling\. Random sampling is unstable \(SR oscillates15\.76→30\.54→18\.7215\.76\\rightarrow 30\.54\\rightarrow 18\.72\)\. Hard\-case selection focuses on original\-success / weak\-failure episodes and yields stronger supervision\. Under hard\-case selection, repeatedly training on a fixed full subset saturates, while resamplingK=20K\{=\}20from a 100\-case pool continues to improve through Round 5 and yields the TGIT result in Table[1](https://arxiv.org/html/2610.10635#S4.T1)\(37\.9337\.93/24\.8524\.85/72\.0872\.08\)\. Selection without resampling peaks early, while resampling without selection remains unstable; their combination supplies relevance and round\-to\-round diversity\.

#### Mechanism: hard cases, failure stage, and visual context\.

Figure[3](https://arxiv.org/html/2610.10635#S4.F3)presents the three oral mechanism slots under a Table[1](https://arxiv.org/html/2610.10635#S4.T1)\-aligned reading\. Selected\-Pool improves through Round 5 to the TGIT result in Table[1](https://arxiv.org/html/2610.10635#S4.T1)\(37\.9337\.93\), while Random\-Full oscillates and Selected\-Full saturates\. Weak failures are dominated by early collapse \(153/203153/203\); SFT recovers to29\.55%29\.55\\%SR but leaves a large early\-fail mass; TGIT expands success to37\.93%37\.93\\%and shrinks early failure\. Frame diagnostics approach the same TGIT numbers \(37\.9337\.93/24\.8524\.85/72\.0872\.08\)\. Across OpenFly/CityNav/AirVLN, relative activation with sharply different absolute levels separates interface repair from the backbone ceiling\.

#### Reward ranking\.

Table[3](https://arxiv.org/html/2610.10635#S4.T3)holds the Round 5 candidate set fixed and varies only how rollout metrics induce pairwise preferences\. It diagnoses ranking robustness among alternatives to the main Weighted form, not a substitute for the TGIT result in Table[1](https://arxiv.org/html/2610.10635#S4.T1)\(37\.93%37\.93\\%\), which also resamples hard cases and chains adapters across rounds\. All success\-prioritized forms recover well above Weak: Success\-only reaches35\.47%35\.47\\%, and lexicographic Success→\\rightarrowSPL→\\rightarrowNE reaches36\.14%36\.14\\%, showing that the activation conclusion is not tied to the exact0\.5/0\.3/0\.20\.5/0\.3/0\.2scalar weights\. We keep Weighted as the main\-protocol reward because it supplies dense efficiency and distance signals when candidates are near\-tied or unsuccessful; the diagnostic alternatives show that switching among success\-first rankings does not reverse the preference\-learning story\.

Table 3:Reward Ranking\(N=100N\{=\}100, Round 5\)\. Alternatives to the main Weighted form on a fixed candidate set\. Main TGIT SR remains37\.93%37\.93\\%\(Table[1](https://arxiv.org/html/2610.10635#S4.T1)\)\.
#### Translator backbone and scale\.

Table[4](https://arxiv.org/html/2610.10635#S4.T4)compares translator backbones under the same diagnosticN=100N\{=\}100,M=4M\{=\}4Round 5 protocol\. Qwen2\.5\-VL\-7B is cleanest \(36\.45%36\.45\\%SR, full coverage\); 2B collapses valid outputs \(156/203156/203\), so absolute SR becomes unreliable under a fixed203203\-episode denominator\. Because model families also differ \(Qwen2\.5/3/3\.5\), this is a practical backbone comparison rather than a monotonic parameter\-scaling law\. It supports 7B as the clean default translator rather than proposing another end\-to\-end agent\.

Table 4:Translator backbone/scale comparison\(N=100N\{=\}100,M=4M\{=\}4; diagnostic\)\. SR uses fixed denominator203203; invalid outputs count as failures\.†NE is valid\-only\. TGIT in Table[1](https://arxiv.org/html/2610.10635#S4.T1)remains the primary result\.
#### Discussion and claim boundaries\.

Hard\-case selection isolates language as the changed variable when Orig succeeds and Weak fails; random/full ablations show that structure and resampling both matter\. Human SR11\.33%11\.33\\%shows the mismatch is not an LM\-weakening artifact\([Dorbala et al\. 2024](https://arxiv.org/html/2610.10635#bib.bib40)\), while Weak remains a stress protocol rather than a Human\-distribution estimator; TGIT\-on\-Human is unmeasured\. TGIT’s main gain is success coverage under the weighted reward \(\+22\.66\+22\.66SR over Weak;\+8\.38\+8\.38over SFT\), not lockstep SPL/NE improvement\. OpenFly shows the strongest activation; CityNav/AirVLN expose the backbone ceiling\.

## 5Conclusion

We studied an instruction\-interface gap in aerial VLN: frozen agents can fail under short, underspecified inputs even when they possess the required navigation skill\. Human annotations establish the mismatch, while Weak provides a controlled stress protocol\.TGITcloses the interface loop through hard\-case selection, trajectory\-ranked preferences, and pool resampling while keeping the UAV agent frozen\. It raises OpenFly Selected\-Seen SR from15\.27%15\.27\\%to37\.93%37\.93\\%, exceeding Orig \(31\.03%31\.03\\%\), ICL, and SFT\. CityNav/AirVLN expose the boundary: translation activates demonstrated skill but cannot strengthen the backbone\.

#### Limitations and future work\.

TGIT currently performs one\-shot translation before navigation and does not revise commands online as observations or execution states change\. A natural next step is latency\-aware, real\-time closed\-loop translation that updates executable instructions during flight\. Moreover, the current agents remain limited in perception and control; another direction is to directly strengthen the navigation backbone itself\.

## References

- Anet al\.\(2023\)G\. An, J\. Lee, X\. Zuo, N\. Kosaka, K\. Kim, and H\. O\. SongDirect preference\-based policy optimization without reward modeling\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px3.p1.1)\.
- Andersonet al\.\(2021\)P\. Anderson, A\. Shrivastava, J\. Truong, A\. Majumdar, D\. Parikh, D\. Batra, and S\. LeeSim\-to\-real transfer for vision\-and\-language navigation\.InConference on Robot Learning,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1),[§4\.2](https://arxiv.org/html/2610.10635#S4.SS2.SSS0.Px3.p1.1)\.
- Andersonet al\.\(2018\)P\. Anderson, Q\. Wu, D\. Teney, J\. Bruce, M\. Johnson, N\. Sünderhauf, I\. Reid, S\. Gould, and A\. van den HengelVision\-and\-language navigation: interpreting visually\-grounded navigation instructions in real environments\.InProceedings of the IEEE Conference on Computer Vision and Pattern Recognition \(CVPR\),pp\. 3674–3683\.Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1),[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px4.p1.2),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.SSS0.Px3.p4.1)\.
- Calandrielloet al\.\(2024\)D\. Calandriello, D\. Guo, R\. Munos, M\. Rowland, Y\. Tang, B\. A\. Pires, P\. H\. Richemond, C\. L\. Lan, M\. Valko, T\. Liu,et al\.Human alignment of large language models through online preference optimisation\.InICML,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px5.p1.1)\.
- Danieleet al\.\(2017\)A\. F\. Daniele, M\. Bansal, and M\. R\. WalterNavigational instruction generation as inverse reinforcement learning with neural machine translation\.InProceedings of the 2017 ACM/IEEE International Conference on Human\-Robot Interaction,Cited by:[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px3.p1.1)\.
- Dorbalaet al\.\(2024\)V\. S\. Dorbala, S\. Chowdhury, and D\. ManochaCan llms generate human\-like wayfinding instructions? towards platform\-agnostic embodied instruction synthesis\.InNAACL,Cited by:[§4\.3](https://arxiv.org/html/2610.10635#S4.SS3.SSS0.Px5.p1.1)\.
- Fanet al\.\(2024\)S\. Fan, R\. Liu, W\. Wang, and Y\. YangNavigation instruction generation with bev perception and large language models\.InECCV,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1)\.
- Fenget al\.\(2022\)W\. Feng, T\. Fu, Y\. Lu, and W\. Y\. WangULN: towards underspecified vision\-and\-language navigation\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),pp\. 6394–6412\.Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Gaoet al\.\(2025\)Y\. Gao, C\. Li, Z\. You, J\. Liu, Z\. Li, P\. Chen, Q\. Chen, Z\. Tang, L\. Wang, P\. Yang,et al\.OpenFly: a comprehensive platform for aerial vision\-language navigation\.arXiv preprint arXiv:2502\.18041\.Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§1](https://arxiv.org/html/2610.10635#S1.p4.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.10635#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.10635#S3.SS1.SSS0.Px2.p1.2),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.SSS0.Px3.p4.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.10635#S4.T1.5.1.1.2)\.
- Gopinathanet al\.\(2024\)M\. Gopinathan, M\. Masek, J\. Abu\-Khalaf, and D\. SuterSpatially\-aware speaker for vision\-and\-language navigation instruction generation\.InACL,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Haoet al\.\(2020\)W\. Hao, C\. Li, X\. Li, L\. Carin, and J\. GaoTowards learning a generic agent for vision\-and\-language navigation via pre\-training\.InCVPR,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1)\.
- Huet al\.\(2021\)J\. E\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, and W\. ChenLoRA: low\-rank adaptation of large language models\.ArXivabs/2106\.09685\.External Links:[Link](https://api.semanticscholar.org/CorpusID:235458009)Cited by:[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px5.p1.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.p1.1)\.
- Huanget al\.\(2022\)Z\. Huang, Z\. Shangguan, J\. Zhang, G\. Bar, M\. Boyd, and E\. Ohn\-BarAssister: assistive navigation via conditional instruction generation\.InECCV,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Hunget al\.\(2025\)C\. Hung, N\. Majumder, H\. Deng, L\. Renhang, Y\. Ang, A\. Zadeh, C\. Li, D\. Herremans, Z\. Wang, and S\. PoriaNORA\-1\.5: a vision\-language\-action model trained using world model\- and action\-based preference rewards\.arXiv preprint arXiv:2511\.14659\.Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px3.p1.1)\.
- Kamathet al\.\(2023\)A\. Kamath, P\. Anderson, S\. Wang, J\. Y\. Koh, A\. Ku, A\. Waters, Y\. Yang, J\. Baldridge, and Z\. ParekhA new path: scaling vision\-and\-language navigation with synthetic instructions and imitation learning\.InCVPR,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p4.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Leeet al\.\(2025\)J\. Lee, T\. Miyanishi, S\. Kurita, K\. Sakamoto, D\. Azuma, Y\. Matsuo, and N\. InoueCityNav: a large\-scale dataset for real\-world aerial navigation\.InICCV,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§1](https://arxiv.org/html/2610.10635#S1.p4.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.10635#S3.SS1.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.10635#S3.SS1.SSS0.Px2.p1.2),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.10635#S4.T1.5.1.11.2)\.
- Lianget al\.\(2022\)J\. Liang, W\. Huang, F\. Xia, P\. Xu, K\. Hausman, B\. Ichter, P\. Florence, and A\. ZengCode as policies: language model programs for embodied control\.arXiv preprint arXiv:2209\.07753\.Cited by:[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px3.p1.1)\.
- Linet al\.\(2023\)J\. Lin, H\. Gao, X\. Feng, R\. Xu, C\. Wang, M\. Zhang, L\. Guo, and S\. XuAdvances in embodied navigation using large language models: a survey\.arXiv preprint arXiv:2311\.00530\.Cited by:[§2](https://arxiv.org/html/2610.10635#S2.p1.1)\.
- Linet al\.\(2025\)S\. Lin, Z\. Li, X\. Zhao, G\. Zhou, L\. Wang, R\. Wei, R\. Tang, J\. Li, H\. Wang, J\. Pang,et al\.VLNVerse: a benchmark for vision\-language navigation with versatile, embodied, realistic simulation and evaluation\.arXiv preprint arXiv:2512\.19021\.Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1)\.
- Liuet al\.\(2023\)S\. Liu, H\. Zhang, Y\. Qi, P\. Wang, Y\. Zhang, and Q\. WuAerialvln: vision\-and\-language navigation for uavs\.InICCV,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§1](https://arxiv.org/html/2610.10635#S1.p4.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2610.10635#S3.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.SSS0.Px1.p1.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.SSS0.Px3.p4.1),[§4\.1](https://arxiv.org/html/2610.10635#S4.SS1.p1.1),[Table 1](https://arxiv.org/html/2610.10635#S4.T1.5.1.19.2)\.
- Maet al\.\(2024\)Y\. Ma, Z\. Song, Y\. Zhuang, J\. Hao, and I\. KingA survey on vision\-language\-action models for embodied ai\.arXiv preprint arXiv:2405\.14093\.Cited by:[§2](https://arxiv.org/html/2610.10635#S2.p1.1)\.
- Qiaoet al\.\(2022\)Y\. Qiao, Y\. Qi, Y\. Hong, Z\. Yu, P\. Wang, and Q\. WuHop: history\-and\-order aware pre\-training for vision\-and\-language navigation\.InCVPR,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1)\.
- Rafailovet al\.\(2023\)R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. FinnDirect preference optimization: your language model is secretly a reward model\.InNeurIPS,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px3.p1.1),[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px5.p1.1)\.
- Sautenkovet al\.\(2025\)O\. Sautenkov, Y\. Yaqoot, A\. Lykov, M\. A\. Mustafa, G\. Tadevosyan, A\. Akhmetkazy, M\. A\. Cabrera, M\. Martynov, S\. Karaf, and D\. TsetserukouUAV\-vla: vision\-language\-action system for large scale aerial mission generation\.InACM/IEEE International Conference on Human\-Robot Interaction \(HRI\),Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1)\.
- Saxenaet al\.\(2025\)P\. Saxena, N\. Raghuvanshi, and N\. GoveasUAV\-vln: end\-to\-end vision language guided navigation for uavs\.InEuropean Conference on Mobile Robots \(ECMR\),Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1)\.
- Shahet al\.\(2023\)D\. Shah, B\. Osiński, S\. Levine,et al\.Lm\-nav: robotic navigation with large pre\-trained models of language, vision, and action\.InConference on robot learning,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p4.1)\.
- Songet al\.\(2025\)X\. Song, W\. Chen, Y\. Liu, W\. Chen, G\. Li, and L\. LinTowards long\-horizon vision\-language navigation: platform, benchmark and method\.InCVPR,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1)\.
- Wanget al\.\(2025a\)L\. Wang, X\. Xia, H\. Zhao, H\. Wang, T\. Wang, Y\. Chen, C\. Liu, Q\. Chen, and J\. PangRethinking the embodied gap in vision\-and\-language navigation: a holistic study of physical and visual disparities\.InICCV,Cited by:[§4\.2](https://arxiv.org/html/2610.10635#S4.SS2.SSS0.Px3.p1.1)\.
- Wanget al\.\(2022\)S\. Wang, C\. Montgomery, J\. Orbay, V\. Birodkar, A\. Faust, I\. Gur, N\. Jaques, A\. Waters, J\. Baldridge, and P\. AndersonLess is more: generating grounded navigation instructions from landmarks\.InCVPR,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2025b\)X\. Wang, D\. Yang, Z\. Wang, H\. Kwan, J\. Chen, W\. Wu, H\. Li, Y\. Liao, and S\. LiuTowards realistic uav vision\-language navigation: platform, benchmark, and methodology\.InProceedings of ICLR,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§1](https://arxiv.org/html/2610.10635#S1.p4.1)\.
- Wanget al\.\(2023a\)X\. Wang, W\. Wang, J\. Shao, and Y\. YangLana: a language\-capable navigator for instruction following and generation\.InCVPR,Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Wanget al\.\(2020\)X\. Wang, Q\. Huang, A\. Celikyilmaz, J\. Gao, D\. Shen, Y\. Wang, W\. Y\. Wang, and L\. ZhangVision\-language navigation policy learning and adaptation\.IEEE transactions on pattern analysis and machine intelligence\.Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p4.1)\.
- Wanget al\.\(2023b\)Z\. Wang, J\. Li, Y\. Hong, Y\. Wang, Q\. Wu, M\. Bansal, S\. Gould, H\. Tan, and Y\. QiaoScaling data generation in vision\-and\-language navigation\.InICCV,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2610.10635#S3.SS1.SSS0.Px2.p1.2)\.
- Weiet al\.\(2025\)Z\. Wei, B\. Lin, Y\. Nie, J\. Chen, S\. Ma, H\. Xu, and X\. LiangUnseen from seen: rewriting observation\-instruction using foundation models for augmenting vision\-language navigation\.IEEE Transactions on Neural Networks and Learning Systems\.Note:RAM paradigm; also arXiv:2503\.18065\. Confirm TNNLS volume/pages if available\.Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Xinget al\.\(2025\)S\. Xing, P\. Li, Y\. Wang, R\. Bai, Y\. Wang, C\. Hu, C\. Qian, H\. Yao, and Z\. TuRe\-align: aligning vision language models via retrieval\-augmented direct preference optimization\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing \(EMNLP\),Cited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px3.p1.1)\.
- Yaoet al\.\(2024\)F\. Yao, Y\. Yue, Y\. Liu, X\. Sun, and K\. FuAeroverse: uav\-agent benchmark suite for simulating, pre\-training, finetuning, and evaluating aerospace embodied world models\.arXiv preprint arXiv:2408\.15511\.Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px1.p1.1)\.
- Zhang and Kordjamshidi \(2023\)Y\. Zhang and P\. KordjamshidiVLN\-Trans: translator for the vision and language navigation agent\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(ACL\),pp\. 13219–13233\.Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p4.1),[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Zhanget al\.\(2024\)Y\. Zhang, Z\. Ma, J\. Li, Y\. Qiao, Z\. Wang, J\. Chai, Q\. Wu, M\. Bansal, and P\. KordjamshidiVision\-and\-language navigation today and tomorrow: a survey in the era of foundation models\.TMLR\.Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p1.1),[§2](https://arxiv.org/html/2610.10635#S2.p1.1)\.
- Zhaoet al\.\(2025\)X\. Zhao, X\. Lyu, X\. Chen, and X\. LiT\-araVLN: translator for agricultural robotic agents on vision\-and\-language navigation\.arXiv preprint arXiv:2509\.06644\.Note:GitHub claims ICIC 2026 Oral; using arXiv until camera\-ready venue confirmedCited by:[§2](https://arxiv.org/html/2610.10635#S2.SS0.SSS0.Px2.p1.1)\.
- Zhouet al\.\(2024\)G\. Zhou, Y\. Hong, and Q\. WuNavgpt: explicit reasoning in vision\-and\-language navigation with large language models\.InAAAI,Cited by:[§1](https://arxiv.org/html/2610.10635#S1.p4.1),[§3\.2](https://arxiv.org/html/2610.10635#S3.SS2.SSS0.Px3.p1.1)\.