Native Multilingual Chain-of-Thought Reasoning in Low-Resource Southeast Asian Languages

# Native Multilingual Chain-of-Thought Reasoning in Low-Resource Southeast Asian Languages
Source: [https://arxiv.org/html/2608.00533](https://arxiv.org/html/2608.00533)
###### Abstract

Large Language Models have achieved substantial progress in reasoning capabilities\. Yet in low\-resource native settings, many suffer from cross\-lingual collapse, reverting to English during intermediate steps that require complex logical reasoning\. This presents a cold\-start bottleneck for policy optimization, whereas standard fine\-tuning risks catastrophic forgetting due to cross\-lingual representation drift\. To address these challenges, we introduce the Onramp\-Sequence Cross\-Distillation \(OSCD\), a post\-training algorithm that projects high\-resource reasoning trajectories into low\-resource vocabulary subspaces during generative training rollouts via an integrated translator agentic loop, ensuring the stable and efficient translation of dynamically generated reference samples for fine\-tuning\. This is coupled with joint\-embedding semantic alignment of both reference and target\-language reasoning traces, thereby bridging the pairwise cross\-lingual representational gaps\. Comprehensive evaluations using theAIME25andHMMT25benchmarks demonstrate that OSCD yields up to 3\.2 times overall improvements in native Southeast Asian languages for mathematical reasoning, of which the joint\-embedding semantic alignment component contributes up to 6\.4% improvements in linguistic debiasing over translation\-only baselines\.

Code, Dataset, Model—https://github\.com/SG\-Lim/OSCD

## Introduction

Large Language Models \(LLMs\) have achieved remarkable reasoning capabilities, largely driven by advancements in Chain\-of\-Thought \(CoT\) prompting and Reinforcement Learning \(RL\) \(Weiet al\.[2022](https://arxiv.org/html/2608.00533#bib.bib29);Shaoet al\.[2024](https://arxiv.org/html/2608.00533#bib.bib3)\)\. Nevertheless, these advancements remain overwhelmingly conditioned on high\-resource typographic regimes, predominantly English \(Tranet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib4);Schutet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib5);Baruaet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib11)\)\. Consequently, linguistic regions characterized by severe data deficits are left behind\. Southeast Asia \(SEA\), for instance, represents a population of 671 million but suffers from a lack of training data for regional development of native frontier capabilities\(Loveniaet al\.[2024](https://arxiv.org/html/2608.00533#bib.bib28)\)\. To compensate for this resource asymmetry, multilingual models inherently exhibit an English\-centric bias, leveraging high\-resource languages as a structural anchor for low\-resource problem\-solving\(Schutet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib5)\)\. This creates a fundamental barrier to user accessibility, wherein the ability to interpret the step\-by\-step reasoning trace remains paramount to critical domains such as education and research\. Because the intermediate process of CoT generation reverts systemically to English following complex logical transitions \(Parket al\.[2026](https://arxiv.org/html/2608.00533#bib.bib8);Zhaoet al\.[2026a](https://arxiv.org/html/2608.00533#bib.bib9);Kanget al\.[2026](https://arxiv.org/html/2608.00533#bib.bib10)\), non\-English native speakers are placed at a severe disadvantage, undermining the true value and equity of artificial intelligence\.

![Refer to caption](https://arxiv.org/html/2608.00533v1/x1.png)Figure 1:Pass@5evaluation of large language models on theAIME25benchmark, with linguistic alignment verifications across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x2.png)Figure 2:Graphical representation of the Onramp Sequence Cross\-Distillation \(OSCD\) algorithm\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x3.png)Figure 3:Graphical representation of the agentic loop system for translation of long\-context reasoning traces\.Because many thinking\-enabled models inherently struggle to generate low\-resource native trajectories on complex, multi\-step reasoning tasks \(Figure[1](https://arxiv.org/html/2608.00533#Sx1.F1)\), we identify a critical bottleneck when applying modern RL approaches, particularly policy optimization algorithms relying on pre\-existing knowledge space and reasoning trajectories to optimize sampling efficiency \(Yueet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib30);Kimet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib31)\)\. This presents a cold\-start training problem, wherein absent of successful native samples to trigger sufficient rewards, the model cannot optimize its policy away from high\-resource English pathways\. On the other hand, standard Supervised Fine\-Tuning \(SFT\) or Continued Pre\-Training \(CPT\) on curated multilingual datasets introduces separate challenges, most notably the risk of catastrophic forgetting \(Alexandrovet al\.[2024](https://arxiv.org/html/2608.00533#bib.bib33);Aggarwalet al\.[2024b](https://arxiv.org/html/2608.00533#bib.bib35);Liu and Niehues[2025](https://arxiv.org/html/2608.00533#bib.bib32)\)\. This is largely attributed to the per\-language misalignment of model hidden state representations, causing the mapping of parallel texts with linguistic differences to disjoint embedding subspaces despite semantic similarities \(Aggarwalet al\.[2024a](https://arxiv.org/html/2608.00533#bib.bib34);Limet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib23);Liet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib36)\)\. As a result, naive training without bridging this gap exacerbates representation drift \(Gurgurovet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib6)\), risking instead the established geometry of high\-resource embeddings\.

To address both the representational bottleneck and cold\-start RL dilemma, we propose a training framework that optimizes for semantic equivalence of generative rollouts across different languages, while expanding the knowledge search space for low\-resource multilingual reasoning \(Figure[2](https://arxiv.org/html/2608.00533#Sx1.F2)\)\. Our summary of key contributions are as follows:

- •We introduce OSCD, a novel training framework designed to enable native multilingual CoT reasoning in low\-resource regional languages, specifically SEA languages\. This involves the dynamic, cross\-policy localization of generative rollouts for fine\-tuning, coupled with joint\-embedding semantic alignment bridging the representational differences of parallel texts across different languages\.
- •Extensive experiments on open\-ended, mathematical reasoning tasks demonstrate the efficiency and robustness of our training approach\. We prove that the reasoning capabilities of English\-dominant models can be effectively transferred to low\-resource settings using synthetically localized data, therefore expanding the multilingual knowledge search space required for subsequent RL\.
- •Comprehensive evaluations on theAIME25andHMMT25benchmarks across language variants show that models post\-trained with OSCD outperform existing multilingual SEA model of a comparable scale\. Our training framework preserves model intelligence in high\-resource settings, while successfully addressing systemic linguistic biases that induce unwanted language fallbacks\.

## Related Work

#### Multilingual Reasoning

Multilingual reasoning is a major NLP challenge, particularly for low\-resource languages\(Tranet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib4)\)\. Consequently, models often map decisions into an English\-adjacent latent space before generating target\-language outputs\(Schutet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib5)\)\. This English\-centric bias causes cross\-lingual understanding failures, uneven reasoning quality, and ’cross\-lingual collapse’ where intermediate CoT reverts to English under increased difficulty \(Hwanget al\.[2025](https://arxiv.org/html/2608.00533#bib.bib7);Parket al\.[2026](https://arxiv.org/html/2608.00533#bib.bib8);Zhaoet al\.[2026a](https://arxiv.org/html/2608.00533#bib.bib9);Kanget al\.[2026](https://arxiv.org/html/2608.00533#bib.bib10)\)\.

To bridge this gap, several methods leverage English as an intermediary via translation, code\-switching, cross\-lingual distillation, or parallel fine\-tuning \(Chenet al\.[2024](https://arxiv.org/html/2608.00533#bib.bib15);Kanget al\.[2026](https://arxiv.org/html/2608.00533#bib.bib10);Chaiet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib14);Baruaet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib11);Zhenget al\.[2026](https://arxiv.org/html/2608.00533#bib.bib13)\)\. Native reasoning approaches include mapping problems into language\-agnostic symbolic spaces \(Ranaldi and Pucci[2025](https://arxiv.org/html/2608.00533#bib.bib16)\), causal interventions to subtract language\-specific hidden states \(Zhaoet al\.[2026b](https://arxiv.org/html/2608.00533#bib.bib12)\), and using reinforcement learning for language\-consistency rewards \(Hwanget al\.[2025](https://arxiv.org/html/2608.00533#bib.bib7)\)\. Nevertheless, native target\-language reasoning systematically lags English\-pivoted approaches, suffering from language\-specific generation errors and conceptual misunderstandings \(Baruaet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib11)\)\.

#### Joint\-Embedding Predictive Architecture

The mapping of varying semantic contexts into a shared latent space aligns with the foundational principles of Joint\-Embedding Predictive Architecture \(JEPA\) \(Garridoet al\.[2024](https://arxiv.org/html/2608.00533#bib.bib18);Maeset al\.[2026](https://arxiv.org/html/2608.00533#bib.bib19)\)\.Huanget al\.\([2026a](https://arxiv.org/html/2608.00533#bib.bib20)\)introduced LLM\-JEPA, combining standard autoregression with an embedding\-space prediction loss to align different semantic views of the same knowledge\. Subsequent frameworks leverage this objective to bound hidden\-state trajectories via geometric regularizers \(Huanget al\.[2026b](https://arxiv.org/html/2608.00533#bib.bib27);Yuan[2026](https://arxiv.org/html/2608.00533#bib.bib26)\), decouple latent reasoning from token generation \(Liuet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib25)\), and project query and document embeddings into shared spaces \(Chenet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib24)\)\.

Limet al\.\([2025](https://arxiv.org/html/2608.00533#bib.bib23)\)demonstrated the importance of aligning semantic properties in low\-resource multilingual contexts, without which models typically default to disjoint, less accurate representations\. Along the topic, contrastive learning applied to monolingual English data have been shown to project cross\-lingual representations into a shared, language\-invariant space \(Wanget al\.[2022](https://arxiv.org/html/2608.00533#bib.bib22)\), whereas minimizing cross\-lingual divergence between probabilistic latent variables guides models to capture structured, language\-agnostic semantic representations \(Sherborneet al\.[2023](https://arxiv.org/html/2608.00533#bib.bib21)\)\.

## Methodology

The OSCD framework comprises two components: \(1\) Rollout with Sequence Mutation, of which dynamically synthesizes native\-language reasoning trajectories to populate low\-resource vocabulary subspaces, and \(2\) Multi\-Objective Cross\-Distillation, which enforces sequence\-level semantic alignment while fine\-tuning is carried out on completions with localized reasoning traces\.

### Rollout with Sequence Mutation

Given a promptpptargeting a low\-resource languagel∈ℒl\\in\\mathcal\{L\}, the pipeline initializes by sampling a completion sequencec∼πϕ\(⋅∣p\)c\\sim\\pi\_\{\\phi\}\(\\cdot\\mid p\)from a high\-resource reference modelπϕ\\pi\_\{\\phi\}\. To isolate its intermediate CoT, we decode the token sequence into a text strings=𝒟​\(c\)s=\\mathcal\{D\}\(c\), where𝒟​\(⋅\)\\mathcal\{D\}\(\\cdot\)is the decoding function\. Using a predefined set of structural delimitersΔ=\{δopen,δclose\}\\Delta=\\\{\\delta\_\{\\text\{open\}\},\\delta\_\{\\text\{close\}\}\\\}\(e\.g\., opening and closing think tags\), we partitionssinto a reasoning stringsrs\_\{r\}and an answer stringsas\_\{a\}:

\(sr,sa\)=Split​\(𝒟​\(c\),Δ\)\(s\_\{r\},s\_\{a\}\)=\\text\{Split\}\\big\(\\mathcal\{D\}\(c\),\\Delta\\big\)\(1\)
To localize dynamically the reasoning process into the low\-resource vocabulary subspace𝒱L\\mathcal\{V\}\_\{L\}, an external translation function𝒯\\mathcal\{T\}is then applied to the generated reasoning string, yielding a low\-resource native trace conditioned onll:

sr′=𝒯​\(sr,l\)s^\{\\prime\}\_\{r\}=\\mathcal\{T\}\(s\_\{r\},l\)\(2\)
Thereafter, the text segments are projected back into the discrete token space while preserving topological boundaries of the original completion\. Letℰ∗​\(⋅\)\\mathcal\{E\}^\{\*\}\(\\cdot\)denote the controlled encoding function\. The mutated completion sequencec′c^\{\\prime\}is reconstructed via an ordered concatenation \(⊕\\oplus\) of the re\-encoded segments and their structural delimiters:

c′=δopen⊕ℰ∗​\(sr′\)⊕δclose⊕ℰ∗​\(sa\)c^\{\\prime\}=\\delta\_\{\\text\{open\}\}\\oplus\\mathcal\{E\}^\{\*\}\(s^\{\\prime\}\_\{r\}\)\\oplus\\delta\_\{\\text\{close\}\}\\oplus\\mathcal\{E\}^\{\*\}\(s\_\{a\}\)\(3\)
We constrain the translation function𝒯\\mathcal\{T\}entirely to the reasoning trace, preserving the teacher’s original answer stringsas\_\{a\}verbatim\. This aims to establish a stable anchor, thus guaranteeing that the underlying inferential trajectory remains bound to a known, valid outcome\.

### Multi\-Objective Cross\-Distillation

#### Full\-Sequence Cross\-Entropy

To drive native multilingual acquisition, we apply full\-sequence log\-likelihood \(ℒCE\\mathcal\{L\}\_\{\\text\{CE\}\}\) across the entirety of the student completion, initializing the student modelπθ\\pi\_\{\\theta\}as its reference teacherπϕ\\pi\_\{\\phi\}\. This ensuresπθ\\pi\_\{\\theta\}constructs its intermediate reasoning steps over the target medium𝒱L\\mathcal\{V\}\_\{L\}, while remaining anchored to a stable answer in close proximity to its original distribution:

ℒCE=−1\|c′\|​∑t=0\|c′\|−1log⁡πθ​\(ct′∣p,c<t′\)\\mathcal\{L\}\_\{\\text\{CE\}\}=\-\\frac\{1\}\{\|c^\{\\prime\}\|\}\\sum\_\{t=0\}^\{\|c^\{\\prime\}\|\-1\}\\log\\pi\_\{\\theta\}\(c^\{\\prime\}\_\{t\}\\mid p,c^\{\\prime\}\_\{<t\}\)\(4\)

#### Joint\-Embedding Semantic Alignment

WhileℒCE\\mathcal\{L\}\_\{\\text\{CE\}\}induces a language shift, naively fine\-tuning on translated text risks further isolation and drifting of model representations into language\-specific subspaces\(Limet al\.[2025](https://arxiv.org/html/2608.00533#bib.bib23)\)\. To avert this concern, we introduce a secondary objectiveℒJEPA\\mathcal\{L\}\_\{\\text\{JEPA\}\}, referencing and adapting fromHuanget al\.\([2026a](https://arxiv.org/html/2608.00533#bib.bib20)\)to fulfil our goal of bridging representational differences through latent\-space alignment of specific pairwise tokens\.

LethkTh^\{T\}\_\{k\}andhk′Sh^\{S\}\_\{k^\{\\prime\}\}denote the last\-layer hidden states corresponding to the closing think tag \(δclose\\delta\_\{\\text\{close\}\}\) at token indiceskkandk′k^\{\\prime\}, respectively, conditioned on native reasoning tracer′r^\{\\prime\}for the student modelπθ\\pi\_\{\\theta\}andrrfor teacher modelπϕ\\pi\_\{\\phi\}\. We minimize the cosine distance between these corresponding hidden states as follows:

ℒJEPA=1−hk′S⋅hkT‖hk′S‖​‖hkT‖\\mathcal\{L\}\_\{\\text\{JEPA\}\}=1\-\\frac\{h^\{S\}\_\{k^\{\\prime\}\}\\cdot h^\{T\}\_\{k\}\}\{\\\|h^\{S\}\_\{k^\{\\prime\}\}\\\|\\\|h^\{T\}\_\{k\}\\\|\}\(5\)
We specifically targetδclose\\delta\_\{\\text\{close\}\}, because it serves as an informational bottleneck encapsulating the aggregated semantics of reasoning process prior to answer generation, while preserving token\-level flexibility across different languages so the model is not overconstrained when generatively navigating its internal representations\.

Consequently, the total loss is formalized as a multi\-objective function:

ℒtotal=λCE​ℒCE\+λJEPA​ℒJEPA\\mathcal\{L\}\_\{\\text\{total\}\}=\\lambda\_\{\\text\{CE\}\}\\mathcal\{L\}\_\{\\text\{CE\}\}\+\\lambda\_\{\\text\{JEPA\}\}\\mathcal\{L\}\_\{\\text\{JEPA\}\}\(6\)
whereλCE\\lambda\_\{\\text\{CE\}\}andλJEPA\\lambda\_\{\\text\{JEPA\}\}are hyperparameter weights corresponding to vocabulary acquisition and cross\-lingual semantic alignment, respectively\.

For post\-training optimization, we train models for11epoch in BF16 precision using the AdamW optimizer with a cosine scheduled learning rate of2×10−52\\times 10^\{\-5\}, a0\.10\.1warmup fraction, and a1\.01\.0gradient clip norm\. Training sequences are capped at40964096prompt tokens and81928192completion tokens\. We also keep inference parameters for generative rollouts and reasoning benchmarks consistent with the default recommendations of the respective model developers, using a maximum completion length of8192081920tokens for evaluations\. BothλCE\\lambda\_\{\\text\{CE\}\}andλJEPA\\lambda\_\{\\text\{JEPA\}\}are set to1\.01\.0for simplicity of this study\.

Table 1:Overall ablation performance comparison acrossAIME25,HMMT25benchmarks evaluating Any\-CoT vs Target\-CoT setups, averaged across 3 target languages \(ZH,EN,IN\) comparing different training loss configurations\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x4.png)Figure 4:Training dynamics comparing different training loss configurations usingQwen3\-VL\-8B\-Thinking\.

## Experimentation

The primary objective of this study is to empirically validate our proposed OSCD training framework\. To this end, we design our experiments to address the following core research questions:

1. 1\.Native Reasoning Accuracy:Can the framework demonstrate improved benchmark performance on open\-ended, deterministic mathematical reasoning tasks for models with no prior native Southeast Asian CoT capabilities, conditioned on native language alignment?
2. 2\.Efficiency of Training:Does the framework provide improved training efficiency in terms of its required compute or dataset size? How does it compare to conventional supervised fine\-tuning approaches that optimize multilingual performance on a massive SEA corpora?
3. 3\.Robustness and Generalizability:Is the framework agnostic to model size and model family? Furthermore, does the acquisition of native multilingual CoT reasoning capabilities introduce catastrophic forgetting in high\-resource base languages \(e\.g\., English\)?

### Experimental Setup

We evaluate our proposed approach using the base reasoning modelsSmolLM3\-3B,Qwen3\-4B\-Thinking\-2507, andQwen3\-VL\-8B\-Thinking\. For our training dataset, we extract a lean subset of 70,000 question\-only samples from theOpenMathReasoning\-tircorpus, discarding the accompanied reasoning and answer traces\. Each question is then paired with a user prompt presented natively in one of 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\), explicitly instructing the model to generate its intermediate CoT in that matching language\.

To translate dynamically the long\-context reasoning traces into specific target languages during training, we serve a local instance ofGemma\-SEA\-LION\-v4\-27B\-IT, coupled with an agentic system𝒯\\mathcal\{T\}to optimize throughput and consistency \(Figure[3](https://arxiv.org/html/2608.00533#Sx1.F3)\)\. This offers a functional alternative to rejection sampling on dynamic datasets\. The temperature for translation initializes at0\.00\.0, incrementing by0\.10\.1with each failed attempt \(capped at0\.40\.4\) for up to1010tries\. Temperatures for scoring and language classification were fixed at0\.00\.0to maintain deterministic evaluations\.

We compare our post\-trained models against the base modelsSmolLM3\-3B,Qwen3\-4B\-Thinking\-2507, andQwen3\-VL\-8B\-Thinking, as well as a multilingual modelQwen\-SEA\-LION\-v4\-8B\-VLof comparable size supervised fine\-tuned on 9 million samples across different SEA languages\. We report thepass@5andmean@5scores to proxy reasoning intelligence and consistency, respectively, with standard deviations where applicable\.

Post\-training experiments were conducted on an Ubuntu 22\.04\.5 LTS platform comprising four NVIDIA H200 GPUs \(141,GB HBM3e VRAM\) interconnected via NVLink\. The software stack consists of PyTorch 2\.9\.0, DeepSpeed 0\.18\.4, HuggingFace Transformers 4\.57\.1, and vLLM 0\.13\.0\. Total GPU runtime accumulated to 384 hours forSmolLM3\-3B, 440 hours forQwen3\-4B\-Thinking\-2507, and 625 hours forQwen3\-VL\-8B\-Thinking, each inclusive of model training alongside the dedicated translator server\.

### Evaluation Metrics

To evaluate the native reasoning capabilities of LLMs across low\-resource Southeast Asian languages, we introduce a comprehensive evaluation framework using theAIME25andHMMT25mathematical reasoning benchmarks \(Zhang and Math\-AI[2025](https://arxiv.org/html/2608.00533#bib.bib1);Dekonincket al\.[2026](https://arxiv.org/html/2608.00533#bib.bib2)\), each translated into 7 different languages \(ZH,EN,Fi,IN,TA,TH,VI\) to accomodate a lack of multilingual benchmarks comprising all target variants\. The framework comprises two scoring modes, to evaluate both a model’s linguistic biases and instruction\-following capabilities\. Both modes evaluate on the same exact generated completions:

- •Any\-CoT:In this mode, the model is scored solely on the correctness of its final parsed answer, regardless if its reasoning CoT exhibits linguistic drift to English or non\-target languages\.
- •Target\-CoT:In this mode, the model is scored on the correctness of its final parsed answer, as well as its adherence to the target languages during CoT reasoning per user instructions\.

To analyze linguistic bias, we also define Linguistic Retention Index\(LRI\)as the probability a model successfully maintains its logical derivation process natively using the target language, rather than collapsing into a high\-resource fallback\. Given a transition matrixM∈ℝN×NM\\in\\mathbb\{R\}^\{N\\times N\}, where the diagonal elementMi,iM\_\{i,i\}denotes the probability that a model with target prompt languageℓi\\ell\_\{i\}maintains its reasoning natively withinℓi\\ell\_\{i\}, theLRIacross a set of target languages𝒢\\mathcal\{G\}is computed as:

LRI=100\|𝒢\|​∑ℓi∈𝒢Mi,i\\text\{LRI\}=\\frac\{100\}\{\|\\mathcal\{G\}\|\}\\sum\_\{\\ell\_\{i\}\\in\\mathcal\{G\}\}M\_\{i,i\}\(7\)
Thereafter, a high LRI for a given\|𝒢\|\|\\mathcal\{G\}\|languages bounded between0and100\(inclusive\) reflects native reasoning capabilities robust from biased linguistic preferences, whereas a low LRI signals systemic fallback behaviour into non\-target languages\.

For linguistic alignment verification of intermediate reasoning steps, we implement LLM\-as\-a\-judge using the modelGemma\-SEA\-LION\-v4\-27B\-IT\. We also truncate the first 70% and tail 10% of sequence lengths, leaving only a 20% critical snippet of reasoning traces for classification\. This prevents false positives induced by data leakage, often caused by model regurgitation of the native prompts near the beginning and end of its reasoning completion\. If language\-mixing or switching were detected, the classification defaults strictly to high\-resource languages, prioritizing English\.

### Ablation Studies

We conduct ablation studies using the base reasoning modelQwen3\-VL\-8B\-Thinking, along with 4 distinct training configurations to isolate the core contributions of individual components within the OSCD framework, specifically the secondary cosine distance lossℒJEPA\\mathcal\{L\}\_\{\\text\{JEPA\}\}as well as the agentic translator system𝒯\\mathcal\{T\}\. TheℒCE\\mathcal\{L\}\_\{\\text\{CE\}\}configuration accounts for the non\-agentic deterministic translation of fine\-tuned reasoning traces as a baseline referencingChenet al\.[2024](https://arxiv.org/html/2608.00533#bib.bib15)andBaruaet al\.[2026](https://arxiv.org/html/2608.00533#bib.bib11), less rejection sampling due to the dynamic nature of data localization involved\. For efficiency of experimentation, we use a representative, downscaled variant of the primary training dataset\. We construct a balanced subset of 7,500 samples, interleaved and uniformly distributed across 3 target languages \(ZH,EN,IN\) to enable a streamlined analysis of performance trade\-offs\.

Accordingly, theℒCE\+JEPA \(Agentic\)\\mathcal\{L\}\_\{\\text\{CE\+JEPA \(Agentic\)\}\}configuration outperforms its other ablation variants, yielding the highest Target\-CoT and LRI scores across both benchmarks \(Table[1](https://arxiv.org/html/2608.00533#Sx3.T1)\)\. Without the agentic translator to suppress zero\-shot translation artifacts and variance, we observe increased noise in its respective gradient norms that led to severe performance degradations \(Figure[4](https://arxiv.org/html/2608.00533#Sx3.F4)b\), in addition to slower inference speeds by a factor of two\. Similarly, without a secondary loss to mitigate cross\-linguistic representational drifts, models consistently suffer from the divergence of its cosine distance metric \(Figure[4](https://arxiv.org/html/2608.00533#Sx3.F4)d\), causing a wider spread in reasoning accuracies and lower LRI scores despite a small set of 3 languages\. This validates both our initial hypothesis and the robustness of our training framework\.

Table 2:Overall main performance comparison acrossAIME25,HMMT25benchmarks evaluating Any\-CoT vs Target\-CoT setups, averaged across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\) comparing base reasoning models and OSCD post\-trained models\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x5.png)Figure 5:Overall main language fallback matrices comparing base reasoning models and OSCD post\-trained models acrossAIME25,HMMT25benchmarks and 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x6.png)Figure 6:3D t\-SNE plots of layer\-wise hidden activations comparing base reasoning models and OSCD post\-trained models, using theAIME25dataset across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\) as input corpus\. The layers11toNNare indicated by expanding marker sizes\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x7.png)Figure 7:3D t\-SNE plots of layer\-wise hidden activations comparing base reasoning models and OSCD post\-trained models, using theHMMT25dataset across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\) as input corpus\. The layers11toNNare indicated by expanding marker sizes\.
### Low\-Resource Native Reasoning

The robustness of model native reasoning performance in low\-resource SEA languages improves substantially with OSCD post\-training\. As shown in Table[2](https://arxiv.org/html/2608.00533#Sx4.T2), we observe an average 2–3 times overall improvement in bothPass@5andMean@5Target\-CoT scores across bothAIME25andHMMT25mathematical benchmarks, reflecting the successful cross\-lingual transfer of high\-resource base capabilities into low\-resource target languages \(Table[2](https://arxiv.org/html/2608.00533#Sx4.T2)\)\. Importantly, the baseline reasoning models exhibit an English sinking state as default fallback language, with a minor exception for Chinese language due to abundance in its pre\-training web corpus \(Figure[5](https://arxiv.org/html/2608.00533#Sx4.F5)\)\. Our models, however, demonstrate consistent mitigation of this bias, achieving a 2\.2–3\.6×\\timesimprovement in LRI relative to model baselines\.

With Any\-CoT, a mild reduction in overall scores for theOurs\-SmolLM3\-3BandOurs\-Qwen3\-VL\-8Bmodels were observed, attributed to the alteration of its natural fallback language\. Isolated for high\-resource languages, however, the performance drops remain marginal compared to baseline \(Figure[1](https://arxiv.org/html/2608.00533#Sx1.F1)\)\. This suggests that foundational reasoning capabilities remain well\-preserved, with noticeable Any\-CoT reductions largely driven by a linguistic shift toward a more unbiased, multilingual distribution\. In contrast,Ours\-Qwen3\-4Bdemonstrated modest improvements under Any\-CoT settings, accompanied by a 40% probability of non\-target fallback behavior; indicating a continued tendency to optimize reasoning trajectories for accuracy via English, while concurrently benefiting from knowledge expansion across low\-resource settings\.

### Training Efficiency and Generalizability

A primary advantage of the OSCD training framework is sample efficiency, which leverages on stable and dynamically localized synthetic completions to mitigate the scarcity of quality data in low\-resource target languages\. To evaluate this, we compare our models againstQwen\-SEA\-LION\-v4\-8B\-VL, a competitive baseline trained on 9 million high\-quality instruction\-text pairs spanning English and SEA languages\. On the other hand, our models utilize a significantly smaller dataset of only 70,000 samples, distributed uniformly across 7 languages\. Despite a raw dataset size reduction of over 90%, our framework yields substantial improvements in LRI metrics \(Table[2](https://arxiv.org/html/2608.00533#Sx4.T2)\)\.Ours\-Qwen3\-VL\-8B, in particular, surpassesQwen\-SEA\-LION\-v4\-8B\-VLin LRI by 30%, thus demonstrating the robustness of dynamic OSCD over static SFT pipelines for multilingual acquisition\.

To evaluate the generalizability of OSCD, we experiment across three parameter scales \(i\.e\., 3B, 4B, and 8B\), spanning two distinct model families \(i\.e\., Qwen3 and SmolLM3\) as well as both text\-only and multimodal dense architectures\. Although our compute constraints prevented scaling beyond 8B parameters, we observed consistent improvements across different models\. Specifically, an average 2–3 times overall improvement in Target\-CoT reasoning performance across different benchmarks, as well as an increase in LRI up to 3\.6 times corresponding to the successfully neutralization of high\-resource English bias present in baseline models\. This provides a working foundation for subsequent post\-training, particularly with open\-ended downstream applications that involve regional and cultural contexts\.

### Scaling Behaviour and Hidden Activations

Our scaling analysis reveals a non\-linear behavior, driven by long\-context reasoning capabilities relative to model weight sizes\.Ours\-SmolLM3\-3B, for instance, lacks the capacity for extended generations which limits its exposure duration to non\-target linguistic fallbacks\. A substantially higher LRI is hence achieved, though its Target\-CoT improvements remain modest\.Ours\-Qwen3\-4B, on the other hand, features extended reasoning capabilities but remains bottlenecked by its compact size, therefore compromising on LRI to sustain the accuracy of its answers\. Among all,Ours\-Qwen3\-VL\-8Bprovides the optimal foundation, supporting the multilingual expansion of knowledge space while simultaneously resisting non\-target linguistic fallbacks over prolonged generations\.

Figures[6](https://arxiv.org/html/2608.00533#Sx4.F6)and[7](https://arxiv.org/html/2608.00533#Sx4.F7)illustrate the layer\-wise hidden\-state activations comparing base reasoning models and OSCD post\-trained models, using theAIME25andHMMT25datasets across 7 target languages as input corpus, respectively\. In general, it is observed the initial layers remain clustered and well\-aligned between models before and after post\-training, while geometric shifts occur predominantly within the middle\-to\-later layers\. The observation reflects a greater dependency on generative steering along deeper layers for downstream benchmark improvements, as opposed to multilingual comprehension across problems in low\-resource languages, particularly with baseline models of larger sizes and stronger default capabilities\. With smaller models, however, substantial shifts can emerge along the early representational layers, indicating a fundamental alteration to how challenging problems are semantically comprehended as new knowledge gets ingested\.

## Conclusion

To conclude, we present the OSCD algorithm which enables 2–3 times substantial improvements in Target\-CoT reasoning performance, conditioned on linguistic alignment verifications across low\-resource Southeast Asian languages\. The enabling of native multilingual reasoning capabilities proved largely additive, with limited observations of catastrophic forgetting observed in high\-resource base capabilities\. Crucially, we extend the upper\-bound intelligence potential for native CoT reasoning in low\-resource languages, offering the open community an additional layer of training foundation for subsequent reinforcement learning\. This empowers non\-native English speakers the ability to leverage advanced AI capabilities, thereby improving system usability and broad accessibility\. Future research will focus on integrating reinforcement learning and cultural fine\-tuning for downstream applications beyond deterministic reasoning\. This includes scaling the approach to open\-ended tasks that heavily rely on regional and cultural contexts\.

## Acknowledgments

This work was supported by the DSO\-AISG Incentive Award\. The views expressed are solely those of the authors and do not represent the opinions of DSO National Laboratories or AI Singapore\.

## References

- D\. Aggarwal, S\. Damle, N\. Goyal, S\. Lokam, and S\. Sitaram \(2024a\)Towards exploring continual fine\-tuning for enhancing language ability in large language model\.InNeurIPS 2024 Workshop on Fine\-Tuning in Modern Machine Learning: Principles and Scalability,External Links:[Link](https://openreview.net/forum?id=1rfmxGtSjq)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- D\. Aggarwal, A\. Sathe, I\. Watts, and S\. Sitaram \(2024b\)MAPLE: multilingual evaluation of parameter efficient finetuning of large language models\.InFindings of the Association for Computational Linguistics: ACL 2024,L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 14824–14867\.External Links:[Link](https://aclanthology.org/2024.findings-acl.881/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-acl.881)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- A\. Alexandrov, V\. Raychev, M\. N\. Müller, C\. Zhang, M\. Vechev, and K\. Toutanova \(2024\)Mitigating catastrophic forgetting in language transfer via model merging\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 17167–17186\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.1000/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.1000)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- J\. Barua, S\. Eisape, K\. Yin, and A\. Suhr \(2026\)Long chain\-of\-thought reasoning across languages\.External Links:2508\.14828,[Link](https://arxiv.org/abs/2508.14828)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1),[Ablation Studies](https://arxiv.org/html/2608.00533#Sx4.SSx3.p1.3)\.
- L\. Chai, J\. Yang, T\. Sun, H\. Guo, J\. Liu, B\. Wang, X\. Liang, J\. Bai, T\. Li, Q\. Peng, and Z\. Li \(2025\)XCoT: cross\-lingual instruction timing for cross\-lingual chain\-of\-thought reasoning\.InProceedings of the Thirty\-Ninth AAAI Conference on Artificial Intelligence and Thirty\-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence,AAAI’25/IAAI’25/EAAI’25\.External Links:ISBN 978\-1\-57735\-897\-8,[Link](https://doi.org/10.1609/aaai.v39i22.34524),[Document](https://dx.doi.org/10.1609/aaai.v39i22.34524)Cited by:[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1)\.
- N\. Chen, Z\. Zheng, N\. Wu, M\. Gong, D\. Zhang, and J\. Li \(2024\)Breaking language barriers in multilingual mathematical reasoning: insights and observations\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 7001–7016\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.411/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.411)Cited by:[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1),[Ablation Studies](https://arxiv.org/html/2608.00533#Sx4.SSx3.p1.3)\.
- T\. Chen, S\. Xu, F\. Guo, X\. Wang, Q\. Gu, H\. Li, and L\. Ji \(2026\)Efficient rationale\-based retrieval: on\-policy distillation from generative rerankers based on jepa\.InProceedings of the 2026 International Conference on Multimedia Retrieval,ICMR ’26,pp\. 1721–1729\.External Links:[Link](http://dx.doi.org/10.1145/3805622.3810780),[Document](https://dx.doi.org/10.1145/3805622.3810780)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1)\.
- J\. Dekoninck, N\. Jovanović, T\. Gehrunger, K\. Rögnvaldsson, I\. Petrov, C\. Sun, and M\. Vechev \(2026\)Beyond benchmarks: matharena as an evaluation platform for mathematics with llms\.External Links:2605\.00674,[Link](https://arxiv.org/abs/2605.00674)Cited by:[Evaluation Metrics](https://arxiv.org/html/2608.00533#Sx4.SSx2.p1.1)\.
- Q\. Garrido, M\. Assran, N\. Ballas, A\. Bardes, L\. Najman, and Y\. LeCun \(2024\)Learning and leveraging world models in visual representation learning\.External Links:2403\.00504,[Link](https://arxiv.org/abs/2403.00504)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1)\.
- D\. Gurgurov, T\. Baeumel, J\. van Genabith, and S\. Ostermann \(2026\)Sparse subnetwork enhancement for underrepresented languages in large language models\.External Links:2510\.13580,[Link](https://arxiv.org/abs/2510.13580)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- H\. Huang, Y\. LeCun, and R\. Balestriero \(2026a\)LLM\-JEPA: large language models meet joint embedding predictive architectures\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=GbXKPo9QfH)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1),[Joint\-Embedding Semantic Alignment](https://arxiv.org/html/2608.00533#Sx3.SSx2.SSSx2.p1.2)\.
- H\. Huang, Y\. LeCun, and R\. Balestriero \(2026b\)Semantic tube prediction: beating LLM data efficiency with JEPA\.InThe 1st Workshop on Scaling Post\-training for LLMs,External Links:[Link](https://openreview.net/forum?id=oPGmawJ5uH)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1)\.
- J\. Hwang, K\. Tanmay, S\. Lee, A\. Agrawal, H\. Palangi, K\. Ayush, I\. Fiete, and P\. P\. Liang \(2025\)Learn globally, speak locally: bridging the gaps in multilingual reasoning\.External Links:2507\.05418,[Link](https://arxiv.org/abs/2507.05418)Cited by:[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1)\.
- D\. Kang, S\. Hwang, D\. Kim, H\. Kim, and G\. G\. Lee \(2026\)Why do multilingual reasoning gaps emerge in reasoning language models?\.External Links:2510\.27269,[Link](https://arxiv.org/abs/2510.27269)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1)\.
- M\. Kim, A\. Shrestha, S\. Shrestha, A\. Nepal, and K\. W\. Ross \(2025\)RLVR vs\. distillation: understanding accuracy and capability in LLM mathematical reasoning\.InThe 5th Workshop on Mathematical Reasoning and AI at NeurIPS 2025,External Links:[Link](https://openreview.net/forum?id=DH9hjro5eu)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- Z\. Li, Y\. Shi, Z\. Liu, F\. Yang, A\. Payani, N\. Liu, and M\. Du \(2025\)Language ranker: a metric for quantifying llm performance across high and low\-resource languages\.InProceedings of the Thirty\-Ninth AAAI Conference on Artificial Intelligence and Thirty\-Seventh Conference on Innovative Applications of Artificial Intelligence and Fifteenth Symposium on Educational Advances in Artificial Intelligence,AAAI’25/IAAI’25/EAAI’25\.External Links:ISBN 978\-1\-57735\-897\-8,[Link](https://doi.org/10.1609/aaai.v39i27.35038),[Document](https://dx.doi.org/10.1609/aaai.v39i27.35038)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- Z\. W\. Lim, A\. F\. Aji, and T\. Cohn \(2025\)Language\-specific latent process hinders cross\-lingual performance\.External Links:2505\.13141,[Link](https://arxiv.org/abs/2505.13141)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1),[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p2.1),[Joint\-Embedding Semantic Alignment](https://arxiv.org/html/2608.00533#Sx3.SSx2.SSSx2.p1.2)\.
- B\. K\. Liu, Z\. P\. Chen, and D\. P\. Woodruff \(2026\)JEPA\-reasoner: decoupling latent reasoning from token generation\.External Links:2512\.19171,[Link](https://arxiv.org/abs/2512.19171)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1)\.
- D\. Liu and J\. Niehues \(2025\)Conditions for catastrophic forgetting in multilingual translation\.InProceedings of the 5th Workshop on Multilingual Representation Learning \(MRL 2025\),D\. I\. Adelani, C\. Arnett, D\. Ataman, T\. A\. Chang, H\. Gonen, R\. Raja, F\. Schmidt, D\. Stap, and J\. Wang \(Eds\.\),Suzhuo, China,pp\. 347–359\.External Links:[Link](https://aclanthology.org/2025.mrl-main.23/),[Document](https://dx.doi.org/10.18653/v1/2025.mrl-main.23),ISBN 979\-8\-89176\-345\-6Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- H\. Lovenia, R\. Mahendra, S\. M\. Akbar, L\. J\. V\. Miranda, J\. Santoso, E\. Aco, A\. Fadhilah, J\. Mansurov, J\. M\. Imperial, O\. P\. Kampman, J\. R\. A\. Moniz, M\. R\. S\. Habibi, F\. Hudi, R\. Montalan, R\. Ignatius, J\. A\. Lopo, W\. Nixon, B\. F\. Karlsson, J\. Jaya, R\. Diandaru, Y\. Gao, P\. Amadeus, B\. Wang, J\. C\. B\. Cruz, C\. Whitehouse, I\. H\. Parmonangan, M\. Khelli, W\. Zhang, L\. Susanto, R\. A\. Ryanda, S\. L\. Hermawan, D\. J\. Velasco, M\. D\. A\. Kautsar, W\. F\. Hendria, Y\. Moslem, N\. Flynn, M\. F\. Adilazuarda, H\. Li, J\. Lee, R\. Damanhuri, S\. Sun, M\. R\. Qorib, A\. Djanibekov, W\. Q\. Leong, Q\. V\. Do, N\. Muennighoff, T\. Pansuwan, I\. F\. Putra, Y\. Xu, T\. N\. Chia, A\. Purwarianti, S\. Ruder, W\. Tjhi, P\. Limkonchotiwat, A\. F\. Aji, S\. Keh, G\. I\. Winata, R\. Zhang, F\. Koto, Z\. Yong, and S\. Cahyawijaya \(2024\)SEACrowd: a multilingual multimodal data hub and benchmark suite for Southeast Asian languages\.InProceedings of the 2024 Conference on Empirical Methods in Natural Language Processing,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 5155–5203\.External Links:[Link](https://aclanthology.org/2024.emnlp-main.296/),[Document](https://dx.doi.org/10.18653/v1/2024.emnlp-main.296)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1)\.
- L\. Maes, Q\. L\. Lidec, D\. Scieur, Y\. LeCun, and R\. Balestriero \(2026\)LeWorldModel: stable end\-to\-end joint\-embedding predictive architecture from pixels\.External Links:2603\.19312,[Link](https://arxiv.org/abs/2603.19312)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1)\.
- C\. Park, J\. Kim, J\. Lee, S\. Bae, J\. Choo, and K\. M\. Yoo \(2026\)Cross\-lingual collapse: how language\-centric foundation models shape reasoning in large language models\.External Links:2506\.05850,[Link](https://arxiv.org/abs/2506.05850)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p1.1)\.
- L\. Ranaldi and G\. Pucci \(2025\)Multilingual reasoning via self\-training\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 11566–11582\.External Links:[Link](https://aclanthology.org/2025.naacl-long.577/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.577),ISBN 979\-8\-89176\-189\-6Cited by:[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1)\.
- L\. Schut, Y\. Gal, and S\. Farquhar \(2025\)Do multilingual LLMs think in english?\.InICLR 2025 Workshop on Building Trust in Language Models and Applications,External Links:[Link](https://openreview.net/forum?id=I8BOtOPcOv)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.CoRRabs/2402\.03300\.External Links:[Link](https://doi.org/10.48550/arXiv.2402.03300)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1)\.
- T\. Sherborne, T\. Hosking, and M\. Lapata \(2023\)Optimal transport posterior alignment for cross\-lingual semantic parsing\.Transactions of the Association for Computational Linguistics11,pp\. 1432–1450\.External Links:[Link](https://aclanthology.org/2023.tacl-1.81/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00611)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p2.1)\.
- K\. Tran, B\. O’Sullivan, and H\. D\. Nguyen \(2026\)Reasoning transfer for an extremely low\-resource and endangered language: bridging languages through sample\-efficient language understanding\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 33277–33286\.Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p1.1)\.
- Y\. Wang, A\. Wu, and G\. Neubig \(2022\)English contrastive learning can learn universal cross\-lingual sentence embeddings\.InProceedings of the 2022 Conference on Empirical Methods in Natural Language Processing,Y\. Goldberg, Z\. Kozareva, and Y\. Zhang \(Eds\.\),Abu Dhabi, United Arab Emirates,pp\. 9122–9133\.External Links:[Link](https://aclanthology.org/2022.emnlp-main.621/),[Document](https://dx.doi.org/10.18653/v1/2022.emnlp-main.621)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p2.1)\.
- J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, B\. Ichter, F\. Xia, E\. H\. Chi, Q\. V\. Le, and D\. Zhou \(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InProceedings of the 36th International Conference on Neural Information Processing Systems,NIPS ’22,Red Hook, NY, USA\.External Links:ISBN 9781713871088Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1)\.
- Y\. Yuan \(2026\)Semantic step prediction: multi\-step latent forecasting in llm reasoning trajectories via step sampling\.External Links:2604\.18464,[Link](https://arxiv.org/abs/2604.18464)Cited by:[Joint\-Embedding Predictive Architecture](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx2.p1.1)\.
- Y\. Yue, Z\. Chen, R\. Lu, A\. Zhao, Z\. Wang, Y\. Yue, S\. Song, and G\. Huang \(2026\)Does reinforcement learning really incentivize reasoning capacity in LLMs beyond the base model?\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=4OsgYD7em5)Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p2.1)\.
- Y\. Zhang and T\. Math\-AI \(2025\)American invitational mathematics examination \(aime\) 2025\.Cited by:[Evaluation Metrics](https://arxiv.org/html/2608.00533#Sx4.SSx2.p1.1)\.
- R\. Zhao, Y\. Liu, H\. Schuetze, and M\. A\. Hedderich \(2026a\)A comprehensive evaluation of multilingual chain\-of\-thought reasoning: performance, consistency, and faithfulness across languages\.InFindings of the Association for Computational Linguistics: EACL 2026,V\. Demberg, K\. Inui, and L\. Marquez \(Eds\.\),Rabat, Morocco,pp\. 5223–5247\.External Links:[Link](https://aclanthology.org/2026.findings-eacl.276/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-eacl.276),ISBN 979\-8\-89176\-386\-9Cited by:[Introduction](https://arxiv.org/html/2608.00533#Sx1.p1.1),[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p1.1)\.
- W\. Zhao, J\. Guo, Y\. Deng, T\. Wu, W\. Zhang, Y\. Hu, X\. Sui, Y\. Zhao, W\. Che, B\. Qin, T\. Chua, and T\. Liu \(2026b\)When less language is more: language\-reasoning disentanglement makes LLMs better multilingual reasoners\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=fleQlZ2VTx)Cited by:[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1)\.
- W\. Zheng, X\. Huang, Z\. Liu, T\. K\. Vangani, B\. Zou, X\. Tao, Y\. Wu, A\. Aw, N\. F\. Chen, and R\. K\. Lee \(2026\)Adamcot: rethinking cross\-lingual factual reasoning through adaptive multilingual chain\-of\-thought\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 33863–33871\.Cited by:[Multilingual Reasoning](https://arxiv.org/html/2608.00533#Sx2.SS0.SSSx1.p2.1)\.

## Appendix ATraining Details

### Algorithm Pseudocode

The details of training process are provided in Algorithm[1](https://arxiv.org/html/2608.00533#alg1)\.

### Hyperparameters

Table[3](https://arxiv.org/html/2608.00533#A1.T3)provides the hyperparameter settings for post\-training experiments\. Inference parameters for generative rollouts and reasoning benchmarks were consistent with the default recommendations of the respective model developers\.

HyperparameterValueOptimization:OptimizerAdamWLearning rate2×10−52\\times 10^\{\-5\}Training:Training epochs1PrecisionBF16Gradient clip norm1\.0Learning Rate Schedule:Scheduler typeCosineWarmup fraction0\.1Loss Weights:Cross\-Entropy \(λC​E\\lambda\_\{CE\}\)1\.0JEPA \(λJ​E​P​A\\lambda\_\{JEPA\}\)1\.0Sequence Lengths:Max prompt tokens\(Training\)4096Max completion tokens\(Training\)8192Max completion tokens\(Benchmark\)81920Table 3:Hyperparameter settings\.
### Compute Resources

- •Hardware:4×4\\timesNVIDIA H200 GPUs \(141 GB HBM3e VRAM each\) interconnected via NVLink\.
- •Software:PyTorch 2\.9\.0, DeepSpeed 0\.18\.4, HuggingFace Transformers 4\.57\.1, and vLLM 0\.13\.0\.
- •GPU Runtime:384 hours forSmolLM3\-3B, 440 hours forQwen3\-4B\-Thinking\-2507, and 625 hours forQwen3\-VL\-8B\-Thinking\. \(Note: Total runtime accounts for both the target model training and local vLLM server for dynamic translations\.\)

### Prompt Templates

This appendix provides example prompt templates and scaffolds used within this study\. Specifically, Table[4](https://arxiv.org/html/2608.00533#A2.T4)provides the language classification prompt, Table[5](https://arxiv.org/html/2608.00533#A2.T5)the zero\-shot translation template, and Table[6](https://arxiv.org/html/2608.00533#A2.T6)the quality evaluation framework\. All scaffolds were integrated into our agentic translator during training\. For benchmarks involving language classification, we rely exclusively on Table[4](https://arxiv.org/html/2608.00533#A2.T4)\.

## Appendix BSupplementary Results

### Detailed Benchmarks

This appendix provides benchmark results for both our ablation studies and main experiments\. Tables[7](https://arxiv.org/html/2608.00533#A2.T7)–[8](https://arxiv.org/html/2608.00533#A2.T8)provide the ablation results forAIME25andHMMT25, respectively, with Figures[8](https://arxiv.org/html/2608.00533#A2.F8)–[9](https://arxiv.org/html/2608.00533#A2.F9)illustrating language fallback matrices comparing different model training configurations across the 3 target languages \(ZH,EN,IN\)\. Tables[9](https://arxiv.org/html/2608.00533#A2.T9)–[12](https://arxiv.org/html/2608.00533#A2.T12)provide the main results forAIME25andHMMT25, respectively, with Figures[10](https://arxiv.org/html/2608.00533#A2.F10)–[11](https://arxiv.org/html/2608.00533#A2.F11)illustrating language fallback matrices comparing different model training configurations across the 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.

### Training Dynamics

Figure[12](https://arxiv.org/html/2608.00533#A2.F12)plots the total backpropagated loss, gradient norm, cross\-entropy, and cosine distance metrics for our main training experiments\.

### Hidden Representations

We provide the hidden\-state representations of models before and after OSCD post\-training using t\-distributed Stochastic Neighbor Embedding \(t\-SNE\) visualizations\. Figures[13](https://arxiv.org/html/2608.00533#A2.F13)–[16](https://arxiv.org/html/2608.00533#A2.F16)illustrate the 2D t\-SNE projections across sequential layers, using 300 parallel texts spanning the 7 target SEA languages, subsampled from theFLORES\-Plusdevtestsplit\.

Algorithm 1Onramp Sequence Cross\-Distillation0:Student Model

πθ\\pi\_\{\\theta\}; Reference Model

πϕ\\pi\_\{\\phi\}; Translator System

𝒯\\mathcal\{T\}
0:High\-Resource Base Languages

ℳ=\{EN,ZH\}\\mathcal\{M\}=\\\{\\text\{EN\},\\text\{ZH\}\\\}; Escape Tokens

Δ=\{t<think\>,t</think\>\}\\Delta=\\\{t\_\{\\texttt\{<think\>\}\},t\_\{\\texttt\{</think\>\}\}\\\}
0:Low\-Resource Prompt\-Language Pairs

𝒫=\{\(pi,li\)\}i=1B\\mathcal\{P\}=\\\{\(p\_\{i\},l\_\{i\}\)\\\}\_\{i=1\}^\{B\}, where

ℒ=\{li\}i=1B\\mathcal\{L\}=\\\{l\_\{i\}\\\}\_\{i=1\}^\{B\}and

ℒ∩ℳ=∅\\mathcal\{L\}\\cap\\mathcal\{M\}=\\emptyset
0:Loss Weights:

λCE=1\.0,λJEPA=1\.0\\lambda\_\{\\text\{CE\}\}=1\.0,\\lambda\_\{\\text\{JEPA\}\}=1\.0
1:foreach training stepdo

2:1\. Rollout and Modify Reasoning

3:Sample reference completion IDs

ci∼πϕ\(⋅∣pi\)c\_\{i\}\\sim\\pi\_\{\\phi\}\(\\cdot\\mid p\_\{i\}\)for

i∈\[1,B\]i\\in\[1,B\]
4:for

i=1i=1to

BBdo

5:if

li∈ℳl\_\{i\}\\in\\mathcal\{M\}thencontinue

6:

str\_ri,str\_ai←split\(πθ\.decode\(ci\),delimiters=\{decode\(t\)∣t∈Δ\}\)\\text\{str\\\_\}r\_\{i\},\\text\{str\\\_\}a\_\{i\}\\ \\leftarrow\\text\{split\}\\big\(\\pi\_\{\\theta\}\.\\text\{decode\}\(c\_\{i\}\),\\text\{delimiters\}=\\\{\\text\{decode\}\(t\)\\mid t\\in\\Delta\\\}\\big\)
7:

str\_​ri′←𝒯\.modify​\(str\_​ri,target\_lang=li\)\\text\{str\\\_\}r^\{\\prime\}\_\{i\}\\leftarrow\\mathcal\{T\}\.\\text\{modify\}\(\\text\{str\\\_\}r\_\{i\},\\text\{target\\\_lang\}=l\_\{i\}\)⊳\\trianglerightTranslate reasoning trace

8:

𝐭𝐨𝐤ri′,𝐭𝐨𝐤ai←πθ\.encode​\(\{str\_​ri′,str\_​ai\},add\_special\_tokens=False\)\\mathbf\{tok\}\_\{r^\{\\prime\}\_\{i\}\},\\mathbf\{tok\}\_\{a\_\{i\}\}\\leftarrow\\pi\_\{\\theta\}\.\\text\{encode\}\(\\\{\\text\{str\\\_\}r^\{\\prime\}\_\{i\},\\text\{str\\\_\}a\_\{i\}\\\},\\text\{add\\\_special\\\_tokens=False\}\)
9:

ci′←\(t<think\>​if​t<think\>∈ci​else​∅\)⊕𝐭𝐨𝐤ri′⊕t</think\>⊕𝐭𝐨𝐤aic^\{\\prime\}\_\{i\}\\leftarrow\(t\_\{\\texttt\{<think\>\}\}\\text\{ if \}t\_\{\\texttt\{<think\>\}\}\\in c\_\{i\}\\text\{ else \}\\emptyset\)\\oplus\\mathbf\{tok\}\_\{r^\{\\prime\}\_\{i\}\}\\oplus t\_\{\\texttt\{</think\>\}\}\\oplus\\mathbf\{tok\}\_\{a\_\{i\}\}⊳\\trianglerightReconstruct completion IDs

10:endfor

11:2\. Multi\-Objective Loss Computation

12:Using TrainEngine, compute:

13:Logprobs

PTP^\{T\}and hidden states

HTH^\{T\}for

cic\_\{i\}given

πϕ\\pi\_\{\\phi\}, where

k=pos​\(t</think\>∈ci\)k=\\text\{pos\}\(t\_\{\\texttt\{</think\>\}\}\\in c\_\{i\}\)
14:Logprobs

PSP^\{S\}and hidden states

HSH^\{S\}for

ci′c^\{\\prime\}\_\{i\}given

πθ\\pi\_\{\\theta\}, where

k′=pos​\(t</think\>∈ci′\)k^\{\\prime\}=\\text\{pos\}\(t\_\{\\texttt\{</think\>\}\}\\in c^\{\\prime\}\_\{i\}\)
15:for

i=1i=1to

BBdo

16:Cross\-Entropy

17:

ℒCE\(i\)←−1\|ci′\|​∑t=0\|ci′\|−1log⁡PS​\(ci,t′∣pi,ci,<t′\)\\mathcal\{L\}\_\{\\text\{CE\}\}^\{\(i\)\}\\leftarrow\-\\frac\{1\}\{\|c^\{\\prime\}\_\{i\}\|\}\\sum\_\{t=0\}^\{\|c^\{\\prime\}\_\{i\}\|\-1\}\\log P^\{S\}\(c^\{\\prime\}\_\{i,t\}\\mid p\_\{i\},c^\{\\prime\}\_\{i,<t\}\)⊳\\trianglerightCompletion w/ modified reasoning

18:Cosine Distance

19:

hiS,hiT←Hi=k′S,Hi=kTh^\{S\}\_\{i\},h^\{T\}\_\{i\}\\leftarrow H^\{S\}\_\{i=k^\{\\prime\}\},H^\{T\}\_\{i=k\}
20:

ℒJEPA\(i\)←1−cos\_sim​\(hiS,hiT\)\\mathcal\{L\}\_\{\\text\{JEPA\}\}^\{\(i\)\}\\leftarrow 1\-\\text\{cos\\\_sim\}\(h^\{S\}\_\{i\},h^\{T\}\_\{i\}\)⊳\\trianglerightLetcos\_sim​\(u,v\)=u⋅v‖u‖​‖v‖\\text\{cos\\\_sim\}\(u,v\)=\\frac\{u\\cdot v\}\{\\\|u\\\|\\\|v\\\|\}

21:endfor

22:3\. Gradient Computation and Update

23:

ℒtotal←1B​∑i=1B\(λCE​ℒCE\(i\)\+λJEPA​ℒJEPA\(i\)\)\\mathcal\{L\}\_\{\\text\{total\}\}\\leftarrow\\frac\{1\}\{B\}\\sum\_\{i=1\}^\{B\}\\left\(\\lambda\_\{\\text\{CE\}\}\\mathcal\{L\}\_\{\\text\{CE\}\}^\{\(i\)\}\+\\lambda\_\{\\text\{JEPA\}\}\\mathcal\{L\}\_\{\\text\{JEPA\}\}^\{\(i\)\}\\right\)
24:Update parameters:

θ←θ−η​∇θℒtotal\\theta\\leftarrow\\theta\-\\eta\\nabla\_\{\\theta\}\\mathcal\{L\}\_\{\\text\{total\}\}
25:endfor

Table 4:Prompt template for LLM\-as\-a\-judge classification ofREASONING\_TRACElanguage\.Table 5:Prompt template for zero\-shot translation ofSOURCE\_TEXTfrom source language toTARGET\_LANG\.Table 6:Prompt template for LLM\-as\-a\-judge quality evaluation of parallel textREASONING\_TRACEs\.Table 7:Ablation performance on theAIME25benchmark comparing different training loss configurations, evaluating Any\-CoT vs Target\-CoT setups across 3 target languages \(ZH,EN,IN\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x8.png)Figure 8:Ablation language fallback matrices on theAIME25benchmark comparing different training loss configurations across 3 target languages \(ZH,EN,IN\)\.Table 8:Ablation performance on theHMMT25benchmark comparing different training loss configurations, evaluating Any\-CoT vs Target\-CoT setups across 3 target languages \(ZH,EN,IN\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x9.png)Figure 9:Ablation language fallback matrices on theHMMT25benchmark comparing different training loss configurations across 3 target languages \(ZH,EN,IN\)\.Table 9:Main performance on theAIME25benchmark evaluating Any\-CoT, comparing base reasoning models and OSCD post\-trained models across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.Table 10:Main performance on theHMMT25benchmark evaluating Any\-CoT, comparing base reasoning models and OSCD post\-trained models across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.Table 11:Main performance on theAIME25benchmark evaluating Target\-CoT, comparing base reasoning models and OSCD post\-trained models across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x10.png)Figure 10:Main language fallback matrices comparing base reasoning models and OSCD post\-trained models across theAIME25benchmark and 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.Table 12:Main performance on theHMMT25benchmark evaluating Target\-CoT, comparing base reasoning models and OSCD post\-trained models across 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x11.png)Figure 11:Main language fallback matrices comparing base reasoning models and OSCD post\-trained models across theHMMT25benchmark and 7 target languages \(ZH,EN,Fi,IN,TA,TH,VI\)\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x12.png)\(a\)Ours\-SmolLM3\-3B
![Refer to caption](https://arxiv.org/html/2608.00533v1/x13.png)\(b\)Ours\-Qwen3\-4B
![Refer to caption](https://arxiv.org/html/2608.00533v1/x14.png)\(c\)Ours\-Qwen3\-VL\-8B

Figure 12:Training dynamics for models \(a\)SmolLM3\-3B, \(b\)Qwen3\-4B\-Thinking\-2507, and \(c\)Qwen3\-VL\-8B\-Thinkingusing the OSCD post\-training framework\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x15.png)\(a\)SmolLM3\-3B\(left\) andOurs\-SmolLM3\-3B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x16.png)\(b\)Qwen3\-4B\-Thinking\-2507\(left\) andOurs\-Qwen3\-4B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x17.png)\(c\)Qwen3\-VL\-8B\-Thinking\(left\) andOurs\-Qwen3\-VL\-8B\(right\)

Figure 13:2D t\-SNE visualizations of the first layer \(Layer 1\) multilingual representations for \(a\)SmolLM3\-3B, \(b\)Qwen3\-4B\-Thinking\-2507, and \(c\)Qwen3\-VL\-8B\-Thinkingbefore \(left\) and after \(right\) post\-training with OSCD\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x18.png)\(a\)SmolLM3\-3B\(left\) andOurs\-SmolLM3\-3B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x19.png)\(b\)Qwen3\-4B\-Thinking\-2507\(left\) andOurs\-Qwen3\-4B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x20.png)\(c\)Qwen3\-VL\-8B\-Thinking\(left\) andOurs\-Qwen3\-VL\-8B\(right\)

Figure 14:2D t\-SNE visualizations of the middle layer \(Layer 13\) multilingual representations for \(a\)SmolLM3\-3B, \(b\)Qwen3\-4B\-Thinking\-2507, and \(c\)Qwen3\-VL\-8B\-Thinkingbefore \(left\) and after \(right\) post\-training with OSCD\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x21.png)\(a\)SmolLM3\-3B\(left\) andOurs\-SmolLM3\-3B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x22.png)\(b\)Qwen3\-4B\-Thinking\-2507\(left\) andOurs\-Qwen3\-4B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x23.png)\(c\)Qwen3\-VL\-8B\-Thinking\(left\) andOurs\-Qwen3\-VL\-8B\(right\)

Figure 15:2D t\-SNE visualizations of the middle layer \(Layer 25\) multilingual representations for \(a\)SmolLM3\-3B, \(b\)Qwen3\-4B\-Thinking\-2507, and \(c\)Qwen3\-VL\-8B\-Thinkingbefore \(left\) and after \(right\) post\-training with OSCD\.![Refer to caption](https://arxiv.org/html/2608.00533v1/x24.png)\(a\)SmolLM3\-3B\(left\) andOurs\-SmolLM3\-3B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x25.png)\(b\)Qwen3\-4B\-Thinking\-2507\(left\) andOurs\-Qwen3\-4B\(right\)
![Refer to caption](https://arxiv.org/html/2608.00533v1/x26.png)\(c\)Qwen3\-VL\-8B\-Thinking\(left\) andOurs\-Qwen3\-VL\-8B\(right\)

Figure 16:2D t\-SNE visualizations of the last layer \(Layer 36\) multilingual representations for \(a\)SmolLM3\-3B, \(b\)Qwen3\-4B\-Thinking\-2507, and \(c\)Qwen3\-VL\-8B\-Thinkingbefore \(left\) and after \(right\) post\-training with OSCD\.