Soft Token Alignment for Cross-Lingual Reasoning

# Soft Token Alignment for Cross-Lingual Reasoning
Source: [https://arxiv.org/html/2606.26466](https://arxiv.org/html/2606.26466)
###### Abstract

Multilingual large language models often produce inconsistent reasoning and answers for semantically equivalent prompts in different languages\. Prior work suggests that intermediate representations can be relatively language\-agnostic, but generation becomes increasingly language\-specific as models commit to discrete output tokens\. This is problematic because language\-specific lexical choices can cause semantically equivalent reasoning paths to diverge across languages\. These divergences motivate searching for a cross\-lingual alignment signal that is less tied to any single vocabulary item or script\. We proposeSOLAR, an auxiliary objective for supervised fine\-tuning that alignssoft\-tokenrepresentations across languages, using English as a pivot\. Soft tokens are probability\-weighted mixtures over the vocabulary embeddings, yielding continuous representations that can aggregate information from semantically related tokens across languages\. We then align each non\-English soft\-token summary to its English counterpart in the shared embedding space\. Across four multilingual reasoning benchmarks, SOLAR improves accuracy by up to \+17\.7 points over the base model and \+3\.8 over standard supervised fine\-tuning, with the largest gains on low\-resource languages\. SOLAR also strengthens final\-layer cross\-lingual similarity and substantially reduces language\-cluster separability, suggesting that aligning soft\-token representations helps preserve shared semantic structure during multilingual reasoning\.

## 1Introduction

Large language models \(LLMs\) increasingly serve multilingual users, yet often produce inconsistent reasoning and answers for semantically equivalent prompts in different languages\[[39](https://arxiv.org/html/2606.26466#bib.bib6),[75](https://arxiv.org/html/2606.26466#bib.bib2)\]\. Recent work studying the internal representations of multilingual models\[[57](https://arxiv.org/html/2606.26466#bib.bib4),[27](https://arxiv.org/html/2606.26466#bib.bib81),[37](https://arxiv.org/html/2606.26466#bib.bib74)\]reveals a consistent mechanism: intermediate Transformer layers encode largely language\-agnostic semantics, while final layers become language\-specialized as the model selects discrete tokens in the target language’s script \(e\.g\.,Chinese characters or Thai glyphs\) for generation\. This late\-stage divergence from shared semantics to language\-specific surface forms is hypothesized to contribute to inconsistent reasoning across languages, even when intermediate representations may have converged on the same solution\.

The transition from final\-layer representations to discrete tokens can act as a bottleneck, collapsing probability mass over semantically related tokens into a single language\-specific choice\. Continuous\-space reasoning offers a way to soften this collapse:soft thinking\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\]replaces discrete next\-token choices with probability\-weighted mixtures over vocabulary embeddings\. The resultingsoft tokensaggregate information across the vocabulary and are fed back as input embeddings for the next autoregressive step\. Because soft tokens are not tied to a single token or language, they may retain cross\-lingual semantic content that discrete token selection discards\. However, prior soft\-thinking methods use soft tokens mainly as an inference\-time generation mechanism\. They soften discrete decoding decisions but do not directly encourage semantically equivalent reasoning paths in different languages to stay aligned\. This gap motivates our hypothesis:soft\-token representations can provide a better training\-time signal for cross\-lingual alignment than discrete tokens\.

We proposeSOLAR\(Soft Token Alignment for Cross\-LingualReasoning\), a lightweight auxiliary fine\-tuning objective that aligns soft\-token representations between English and non\-English parallel reasoning traces during supervised fine\-tuning \(SFT\) \(Figure[1](https://arxiv.org/html/2606.26466#S1.F1)\)\. For each reasoning trace and final response, SOLAR summarizes the model’s next\-token distribution as an expectation over the embedding matrix\. It then mean\-pools across positions to obtain a single continuous vector that is independent of any individual token identity\. Using English as a pivot, SOLAR minimizes cosine distance between the English representation and each non\-English counterpart, drawing all languages toward an English\-anchored semantic space\. Because soft tokens operate in the shared embedding space, they enable direct comparison across languages regardless of surface\-form differences\.

Our experiments show that SOLAR improves both task accuracy and cross\-lingual consistency across four multilingual reasoning benchmarks\. For Qwen3\-4B\[[68](https://arxiv.org/html/2606.26466#bib.bib1)\], SOLAR improves accuracy by up to \+3\.8 over SFT and cross\-lingual consistency by up to \+4\.5 on AIME 2024, with the largest gains on low\-resource languages such as Swahili\. Representation analysis shows that soft tokens carry a stronger final\-layer cross\-lingual signal than discrete tokens on base models, and that SOLAR produces substantially stronger final\-layer cross\-lingual alignment than SFT and inference\-time soft thinking alone\. At the same time, behavioral analysis shows that these gains preserve native\-language reasoning: on MGSM, SOLAR\-tuned Qwen3\-4B reasons in the target language’s script at 98\.13% \(vs\. 98\.81% for the base\), and SOLAR\-tuned Qwen3\-8B at 98\.16%\. This reverses the base 8B model’s collapse to English on non\-Chinese languages \(Ja/Th/Te below 5%; 27\.43% four\-language average\)\. To our knowledge, SOLAR is the first method to leverage soft tokens, originally introduced for inference\-time continuous reasoning, in a cross\-lingual alignment objective during post\-training\.

![Refer to caption](https://arxiv.org/html/2606.26466v1/x1.png)Figure 1:SOLAR framework\. Parallel English and non\-English inputs are processed by a shared LLM under teacher forcing, producing a per\-position soft token𝐞tsoft\\mathbf\{e\}^\{\\,\\text\{soft\}\}\_\{t\}that mixes vocabulary embeddings under the model’s next\-token distribution\. Soft tokens are mean\-pooled across response positions into a per\-language summary vector \(𝐞¯ensoft\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\text\{en\}\},𝐞¯ℓsoft\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell\}\)\. The alignment objectiveℒalign\\mathcal\{L\}\_\{\\text\{align\}\}is the cosine distance between the English and non\-English summaries; stop\-gradient on the English side restricts alignment gradients to the non\-English branch\. Both branches are supervised by cross\-entropy \(ℒCE\(en\)\\mathcal\{L\}^\{\(\\text\{en\}\)\}\_\{\\text\{CE\}\},ℒCE\(ℓ\)\\mathcal\{L\}^\{\(\\ell\)\}\_\{\\text\{CE\}\}\), and the non\-English objective addsℒalign\\mathcal\{L\}\_\{\\text\{align\}\}weighted byλ\>0\\lambda\>0, which controls alignment strength\.
## 2SOLAR: soft token alignment for cross\-lingual reasoning

In this section, we first review discrete and soft thinking as two paradigms for constructing input representations during autoregressive reasoning \(§[2\.1](https://arxiv.org/html/2606.26466#S2.SS1)\), then describe how SOLAR repurposes soft\-token embeddings as a cross\-lingual alignment signal during SFT \(§[2\.2](https://arxiv.org/html/2606.26466#S2.SS2)\)\.

### 2\.1Background: discrete vs\. soft thinking

In Chain\-of\-Thought \(CoT\)\[[61](https://arxiv.org/html/2606.26466#bib.bib41)\], which we callDiscrete Thinking, reasoning proceeds by autoregressively generating discrete tokens\. Letx1:nx\_\{1:n\}denote the input prompt tokens andx^n\+1:t−1\\hat\{x\}\_\{n\+1:t\-1\}the previously generated reasoning tokens\. Let𝐄​\[⋅\]\\mathbf\{E\}\[\\cdot\]denote the token embedding lookup function, with embeddings𝐞i=𝐄​\[xi\]\\mathbf\{e\}\_\{i\}=\\mathbf\{E\}\[x\_\{i\}\]fori≤ni\\leq nand𝐞i=𝐄​\[x^i\]\\mathbf\{e\}\_\{i\}=\\mathbf\{E\}\[\\hat\{x\}\_\{i\}\]fori\>ni\>n\. The next token at stepttis decoded frompθ\(⋅∣𝐞1:t−1\)p\_\{\\theta\}\(\\cdot\\mid\\mathbf\{e\}\_\{1:t\-1\}\)\(e\.g\.,greedy or sampling\):

x^t∼pθ\(⋅∣𝐞1:t−1\),𝐞tdiscrete=𝐄\[x^t\]\.\\hat\{x\}\_\{t\}\\sim p\_\{\\theta\}\(\\cdot\\mid\\mathbf\{e\}\_\{1:t\-1\}\),\\qquad\\mathbf\{e\}^\{\\,\\text\{discrete\}\}\_\{t\}=\\mathbf\{E\}\[\\hat\{x\}\_\{t\}\]\.\(1\)
For a language modelfθf\_\{\\theta\}with vocabularyVVand token embedding matrix𝐄∈ℝ\|V\|×d\\mathbf\{E\}\\in\\mathbb\{R\}^\{\|V\|\\times d\}, the logit vector𝐳t=fθ​\(𝐞1:t−1\)∈ℝ\|V\|\\mathbf\{z\}\_\{t\}=f\_\{\\theta\}\(\\mathbf\{e\}\_\{1:t\-1\}\)\\in\\mathbb\{R\}^\{\|V\|\}induces a distributionpθ\(⋅∣𝐞1:t−1\)=softmax\(𝐳t/τ\)p\_\{\\theta\}\(\\cdot\\mid\\mathbf\{e\}\_\{1:t\-1\}\)=\\operatorname\{softmax\}\(\\mathbf\{z\}\_\{t\}/\\tau\), whereτ\\tauis the temperature\. Eq\.[1](https://arxiv.org/html/2606.26466#S2.E1)thus constitutes a hard projection from the distribution over the vocabularyVVto the embedding of a single discrete tokenx^t∈V\\hat\{x\}\_\{t\}\\in V\. That is, regardless of how probability mass is distributed, only𝐄​\[x^t\]\\mathbf\{E\}\[\\hat\{x\}\_\{t\}\]is fed as the input representation for the next step\.

Soft Thinking\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\]is an inference\-time, autoregressive method that replaces this hard discrete projection during reasoning with a continuous representation formed from the next\-token distribution\. At steptt, it defines a*soft\-token*embedding as the expectation over token embeddings𝐄​\[v\]\\mathbf\{E\}\[v\]forv∈Vv\\in Vunder a truncated\-and\-renormalized distribution:

𝐞tsoft=∑v∈Vp~θ​\(v∣𝐞1:t−1\)⋅𝐄​\[v\],\\mathbf\{e\}^\{\\,\\text\{soft\}\}\_\{t\}=\\sum\_\{v\\in V\}\\tilde\{p\}\_\{\\theta\}\(v\\mid\\mathbf\{e\}\_\{1:t\-1\}\)\\cdot\\mathbf\{E\}\[v\],\(2\)wherep~θ\\tilde\{p\}\_\{\\theta\}is constructed by applying top\-kkfiltering to the original distributionpθ​\(v∣𝐞1:t−1\)=\[softmax⁡\(𝐳t/τ\)\]vp\_\{\\theta\}\(v\\mid\\mathbf\{e\}\_\{1:t\-1\}\)=\\left\[\\operatorname\{softmax\}\\\!\\left\(\\mathbf\{z\}\_\{t\}/\\tau\\right\)\\right\]\_\{v\}\. Specifically, letFt⊆VF\_\{t\}\\subseteq Vbe the token set obtained by retaining thekkhighest\-probability tokens\. The filtered distribution is

p~θ​\(v∣𝐞1:t−1\)=pθ​\(v∣𝐞1:t−1\)​𝟏​\[v∈Ft\]∑v′∈Ftpθ​\(v′∣𝐞1:t−1\)\.\\tilde\{p\}\_\{\\theta\}\(v\\mid\\mathbf\{e\}\_\{1:t\-1\}\)=\\frac\{p\_\{\\theta\}\(v\\mid\\mathbf\{e\}\_\{1:t\-1\}\)\\mathbf\{1\}\[v\\in F\_\{t\}\]\}\{\\sum\_\{v^\{\\prime\}\\in F\_\{t\}\}p\_\{\\theta\}\(v^\{\\prime\}\\mid\\mathbf\{e\}\_\{1:t\-1\}\)\}\.\(3\)The resulting𝐞tsoft\\mathbf\{e\}^\{\\,\\text\{soft\}\}\_\{t\}corresponds to no single discrete token and is fed back as the input embedding at stept\+1t\{\+\}1, closing an autoregressive loop over continuous representations instead of discrete tokens\.

### 2\.2Aligning soft tokens for cross\-lingual consistency

#### From inference\-time decoding to training\-time alignment\.

Soft tokens were originally introduced as a single\-language, inference\-time enhancement to CoT\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\], where they are fed back autoregressively during decoding\. For cross\-lingual alignment, their continuous form is useful because each soft token mixes shared vocabulary embeddings rather than committing to a single language\-specific token or script\. Inference\-only soft thinking can soften decoding decisions, but does not explicitly train semantically equivalent reasoning paths across languages to stay aligned\. SOLAR instead uses soft tokens as a training\-time alignment signal during SFT: the forward pass stays teacher\-forced on discrete reference tokens, soft tokens are used only to compute an auxiliary alignment loss, and non\-English soft\-token summaries are pulled toward an English pivot in the shared embedding space\.

#### Soft\-token summary and alignment\.

Given a mini\-batch of parallel input\-response pairs\(x\(ℓ\),y\(ℓ\)\)ℓ∈ℒ\{\(x^\{\(\\ell\)\},y^\{\(\\ell\)\}\)\}\_\{\\ell\\in\\mathcal\{L\}\}across a set of languagesℒ\\mathcal\{L\}\(with Englishen∈ℒ\\text\{en\}\\in\\mathcal\{L\}\), a teacher\-forced forward pass produces logits𝐳t\\mathbf\{z\}\_\{t\}at each response positiont∈\{1,…,T\}t\\in\\\{1,\\dots,T\\\}\. After top\-kkfiltering and temperature scaling,p~θ​\(v∣𝐞1:t−1\)\\tilde\{p\}\_\{\\theta\}\(v\\mid\\mathbf\{e\}\_\{1:t\-1\}\)defines the soft\-token embedding𝐞tsoft\\mathbf\{e\}^\{\\,\\text\{soft\}\}\_\{t\}via Eq\.[2](https://arxiv.org/html/2606.26466#S2.E2)\. For each reasoning trace and final response, we mean\-pool these soft tokens into a summary vector𝐞¯ℓsoft=1T​∑t=1T𝐞tsoft∈ℝd\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell\}=\\frac\{1\}\{T\}\\sum\_\{t=1\}^\{T\}\\mathbf\{e\}^\{\\,\\text\{soft\}\}\_\{t\}\\in\\mathbb\{R\}^\{d\}\. Unlike a discrete token embedding, this vector summarizes likely continuations in the shared embedding space instead of a single vocabulary item or a distribution overℝ\|V\|\\mathbb\{R\}^\{\|V\|\}\. Responses can therefore be compared even across languages with disjoint vocabularies: SOLAR aligns*semantic mixtures of embeddings*, not raw token\-distribution support\.

#### Cross\-lingual alignment objective\.

Given a parallel training pair consisting of responses in two different languagesℓ∗\\ell^\{\*\}andℓ\\ell, we let𝐞¯ℓ∗soft\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell^\{\*\}\}and𝐞¯ℓsoft\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell\}denote the mean\-pooled soft\-token embeddings of the pivot and target responses\. To encourage their response\-level soft\-token summaries to be close in the shared embedding space, we minimize their discrepancy using cosine distance:

ℒalign​\(ℓ∗,ℓ\)=1−cos⁡\(𝐞¯ℓ∗soft,𝐞¯ℓsoft\),\\mathcal\{L\}\_\{\\text\{align\}\}\(\\ell^\{\*\},\\ell\)=1\-\\cos\\\!\\left\(\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell^\{\*\}\},\\;\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell\}\\right\),\(4\)whereℓ∗\\ell^\{\*\}denotes the pivot language\. We form\(ℓ∗,ℓ\)\(\\ell^\{\*\},\\ell\)pairs for every non\-pivot languageℓ\\ell, with the pivot summary𝐞¯ℓ∗soft\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell^\{\*\}\}treated as a fixed target so that alignment gradients flow exclusively through𝐞¯ℓsoft\\bar\{\\mathbf\{e\}\}^\{\\,\\text\{soft\}\}\_\{\\ell\}\. This pulls non\-pivot representations toward the pivot\. We setℓ∗=en\\ell^\{\*\}=\\text\{en\}\(English\) in all experiments, since English consistently yields the highest accuracy across our benchmarks \(Table[2](https://arxiv.org/html/2606.26466#S3.T2)\)\.

#### Integration with supervised fine\-tuning\.

In the SFT setting, the alignment objectiveℒalign\\mathcal\{L\}\_\{\\text\{align\}\}is added as an auxiliary loss for each non\-English training sample in languageℓ\\ell:

ℒSFT\(ℓ\)=ℒCE\(ℓ\)\+λ⋅ℒalign​\(ℓ∗,ℓ\),\\mathcal\{L\}^\{\(\\ell\)\}\_\{\\text\{SFT\}\}=\\mathcal\{L\}^\{\(\\ell\)\}\_\{\\text\{CE\}\}\+\\lambda\\cdot\\mathcal\{L\}\_\{\\text\{align\}\}\(\\ell^\{\*\},\\ell\),\(5\)whereℒCE\(ℓ\)\\mathcal\{L\}^\{\(\\ell\)\}\_\{\\text\{CE\}\}is the cross\-entropy loss for the response in languageℓ\\ell,λ\>0\\lambda\>0controls alignment strength, and soft tokens are computed from the same teacher\-forced forward pass used to evaluateℒCE\\mathcal\{L\}\_\{\\text\{CE\}\}\. For English samples, the alignment term is omitted, leavingℒSFT\(en\)=ℒCE\(en\)\\mathcal\{L\}^\{\(\\text\{en\}\)\}\_\{\\text\{SFT\}\}=\\mathcal\{L\}^\{\(\\text\{en\}\)\}\_\{\\text\{CE\}\}\.

## 3Experiments

We evaluate SOLAR on multilingual reasoning benchmarks to test whether soft\-token alignment improves both accuracy and cross\-lingual consistency\. Further details are in Appendix[D](https://arxiv.org/html/2606.26466#A4)\.

### 3\.1Experimental setup

#### Training\.

We apply supervised fine\-tuning to Qwen3\-4B/8B and Gemma3\-4B/12B\[[53](https://arxiv.org/html/2606.26466#bib.bib42)\]using M\-s1k\[[1](https://arxiv.org/html/2606.26466#bib.bib76)\]\. M\-s1k is a multilingual long CoT reasoning dataset translated from s1k\[[33](https://arxiv.org/html/2606.26466#bib.bib49)\]by Gemini\-2\.0\-Flash\[[15](https://arxiv.org/html/2606.26466#bib.bib25)\]\. s1k consists of 1,000 diverse, high\-quality, and challenging questions paired with complex reasoning traces and solutions\. We compute soft tokens withτ=1\.0\\tau\{=\}1\.0and top\-k=30k\{=\}30\.

#### Evaluation\.

We evaluate on MGSM\[[48](https://arxiv.org/html/2606.26466#bib.bib10)\]and three XReasoning benchmarks\[[38](https://arxiv.org/html/2606.26466#bib.bib20)\]: AIME 2024\[[56](https://arxiv.org/html/2606.26466#bib.bib18)\], AIME 2025\[[22](https://arxiv.org/html/2606.26466#bib.bib19)\], and GPQA Diamond\[[43](https://arxiv.org/html/2606.26466#bib.bib13)\]\(denoted GPQA hereafter\)\. Each benchmark covers 11 languages; we report the 7 overlapping with M\-s1k: En, Fr, Ja, Sw, Te, Th, and Zh\. FollowingZhanget al\.\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\], we decode with temperature0\.60\.6,k=30k\{=\}30,p=0\.95p\{=\}0\.95, and a maximum length of 32,768 tokens\. To reduce variance, we average over 5 runs for MGSM and GPQA and 10 for AIME\.

#### Metrics\.

We report \(1\) Task Accuracy \(↑\\uparrow\): the proportion of questions answered correctly; and \(2\) Cross\-Lingual Consistency \(CLC↑\\uparrow\): the proportion of questions answered correctly in both English and non\-English languages simultaneously\. Inspired byLimet al\.\[[27](https://arxiv.org/html/2606.26466#bib.bib81)\], we define CLC for language pair\(ℓ1,ℓ2\)\(\\ell\_\{1\},\\ell\_\{2\}\)asCLC​\(ℓ1,ℓ2\)=1Nq​∑i=1Nq𝟏​\[aiℓ1=aiℓ2=ai∗\]\\mathrm\{CLC\}\(\\ell\_\{1\},\\ell\_\{2\}\)=\\frac\{1\}\{N\_\{q\}\}\\sum\_\{i=1\}^\{N\_\{q\}\}\\mathbf\{1\}\\\!\\left\[a\_\{i\}^\{\\ell\_\{1\}\}=a\_\{i\}^\{\\ell\_\{2\}\}=a\_\{i\}^\{\*\}\\right\], whereaiℓa\_\{i\}^\{\\ell\}is the model’s answer for theii\-th question in languageℓ\\ell,ai∗a\_\{i\}^\{\*\}is the ground\-truth answer, andNqN\_\{q\}is the total number of questions\.

#### Baselines\.

We compare SOLAR with several approaches: \(1\) base models with standard autoregressive decoding; \(2\) Soft Thinking\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\]: inference\-time replacement of discrete tokens with soft\-token embeddings; \(3\) SFT: standard supervised fine\-tuning without SOLAR’s alignment objective; \(4\) MidAlign\[[29](https://arxiv.org/html/2606.26466#bib.bib60)\]: a middle\-layer representation alignment method based on contrastive learning; \(5\) MAPO\[[46](https://arxiv.org/html/2606.26466#bib.bib55)\]: a preference optimization method aligning non\-English reasoning with English via translation model scores; and \(6\) AlignX\[[2](https://arxiv.org/html/2606.26466#bib.bib7)\]: a two\-stage framework combining intermediate\-layer contrastive alignment with a language matching classifier\.

### 3\.2Main results

Table[1](https://arxiv.org/html/2606.26466#S3.T1)reports aggregate results across four benchmarks, and Table[2](https://arxiv.org/html/2606.26466#S3.T2)breaks down MGSM accuracy by language\. Per\-language results for AIME 2024, AIME 2025, and GPQA are provided in Appendix[E](https://arxiv.org/html/2606.26466#A5)\.

#### SOLAR outperforms all evaluated baselines across benchmarks and model sizes\.

For Qwen3\-4B, SOLAR improves accuracy over SFT by \+1\.5 to \+3\.8 and CLC by \+3\.1 to \+4\.5 across all benchmarks\. On Qwen3\-8B, it achieves \+1\.3 to \+1\.9 accuracy and \+1\.5 to \+2\.9 consistency gains over SFT\. Among cross\-lingual alignment baselines, MidAlign is competitive on the harder benchmarks, reaching second\-best accuracy on AIME 2024 \(36\.67%\) and AIME 2025 \(29\.05%\)\. MAPO is competitive on MGSM \(77\.52%\) but underperforms on AIME 2025 and GPQA, likely due to sparse preference pairs: Qwen3\-4B answers only 1,148 of 9,000 non\-English M\-s1k questions correctly under 20\-sample decoding\. AlignX is comparable to MidAlign and MAPO \(e\.g\.,77\.69% on MGSM\)\.

#### Low\-resource languages benefit most, with the largest gains on Swahili\.

SSwahili accuracy on MGSM nearly doubles from 29\.04% to 53\.48% for Qwen3\-4B and increases from 59\.36% to 81\.36% for Qwen3\-8B, while high\-resource languages maintain stable performance \(Table[2](https://arxiv.org/html/2606.26466#S3.T2)\)\. This reflects the intuition that cross\-lingual alignment is most helpful where cross\-lingual divergence is greatest\.

#### Inference\-time soft thinking does not consistently improve cross\-lingual reasoning\.

For Qwen3\-4B, accuracy changes stay within±\\pm0\.8 points across all benchmarks, and CLC shifts are inconsistent \(\+1\.06 on MGSM but−\-0\.25 on GPQA\)\. This indicates that simply replacing discrete tokens with soft tokens at inference time cannot substitute for explicit training\-time alignment as in SOLAR\.

Table 1:Average accuracy and cross\-lingual consistency \(%\) across four benchmarks for Qwen3\-4B, Qwen3\-8B, and their fine\-tuned variants\.Bestandsecond\-bestwithin each model are highlighted\.MGSMAIME 2024AIME 2025GPQAAvg\.ModelAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCQwen3\-4B74\.0169\.3131\.1023\.8923\.0517\.6127\.5214\.5638\.9231\.34\+ Soft Thinking74\.2370\.3731\.8624\.4423\.3318\.2227\.3414\.3139\.1931\.84\+ SFT76\.2572\.2934\.7625\.1727\.9020\.4443\.7533\.1545\.6737\.76\+ MidAlign76\.4573\.1136\.6727\.1129\.0523\.2243\.0032\.4746\.2938\.98\+ MAPO77\.5273\.9034\.7625\.8925\.8118\.9440\.3230\.1544\.6037\.22\+ AlignX77\.6973\.0036\.0026\.8827\.7620\.1043\.0332\.9746\.1238\.24\+ SOLAR \(ours\)78\.2775\.6138\.5229\.6730\.9524\.5145\.2436\.2648\.2541\.51Qwen3\-8B83\.4280\.5162\.1056\.0654\.9145\.0652\.6842\.6963\.2856\.08\+ Soft Thinking83\.0480\.0162\.5256\.9154\.7144\.2253\.0643\.4263\.3356\.14\+ SFT86\.9084\.7165\.6758\.6758\.3850\.1363\.6455\.7368\.6562\.31\+ SOLAR \(ours\)88\.2787\.2867\.5260\.6160\.1053\.0264\.9657\.2770\.2164\.55

Table 2:Per\-language accuracy \(%\) and average accuracy on MGSM for Qwen3\-4B, Qwen3\-8B, and their fine\-tuned variants\.ModelEnFrJaSwTeThZhAvg\.Qwen3\-4B93\.3681\.3680\.5629\.0464\.2483\.3686\.1674\.01\+ Soft Thinking93\.4083\.5281\.8428\.2461\.6084\.1286\.9274\.23\+ SFT94\.0085\.6077\.2047\.2063\.5279\.3686\.8876\.25\+ MidAlign93\.2485\.1679\.9248\.7263\.0477\.6087\.4476\.45\+ MAPO92\.2485\.7680\.0051\.2865\.9279\.0488\.4077\.52\+ AlignX93\.1284\.5680\.3253\.4864\.3280\.4087\.6077\.69\+ SOLAR \(ours\)93\.3286\.3281\.0053\.4865\.3680\.5687\.8478\.27Qwen3\-8B94\.8886\.8084\.9659\.3680\.0889\.3688\.4883\.42\+ Soft Thinking94\.9686\.8084\.8858\.3281\.4488\.0086\.8883\.04\+ SFT95\.4489\.6082\.2479\.7680\.5691\.2089\.5286\.90\+ SOLAR \(ours\)94\.9691\.2882\.7281\.3683\.2092\.8091\.6088\.27

## 4Analysis

![Refer to caption](https://arxiv.org/html/2606.26466v1/x2.png)Figure 2:t\-SNE of Gemma3\-12B hidden states on MGSM at the middle \(layer 24\) and final \(layer 48\) layers\. Language representations intermix at layer 24 under both thinking modes\. At layer 48, discrete thinking yields separated clusters, while soft thinking substantially increases overlap\.This section analyzes the mechanisms behind SOLAR\. We examine how SOLAR reshapes cross\-lingual representations \(§[4\.1](https://arxiv.org/html/2606.26466#S4.SS1)\), verify that SOLAR preserves target\-language reasoning without shifting generation toward English \(§[4\.2](https://arxiv.org/html/2606.26466#S4.SS2)\), analyze the sensitivity of the alignment weight and soft\-token temperature \(§[4\.3](https://arxiv.org/html/2606.26466#S4.SS3)\), and isolate which component of the objective drives the gain \(§[4\.4](https://arxiv.org/html/2606.26466#S4.SS4)\)\.

### 4\.1Representation analysis

We examine how hidden representations are organized across languages and how soft tokens affect cross\-lingual alignment\. To do so, we first quantify the effect of inference\-time soft thinking \(Table[3](https://arxiv.org/html/2606.26466#S4.T3)\), then ask whether SOLAR further reshapes these representations on Qwen3\-4B \(Table[4](https://arxiv.org/html/2606.26466#S4.T4)\)\. Let𝐡iℓ∈ℝd\\mathbf\{h\}\_\{i\}^\{\\ell\}\\in\\mathbb\{R\}^\{d\}be the hidden state for sampleiiin languageℓ\\ell, with language setℒ\\mathcal\{L\},L=\|ℒ\|L=\|\\mathcal\{L\}\|, andNNparallel samples per language\. We report two metrics in\[−1,\+1\]\[\-1,\+1\]: \(1\)Cross\-Lingual Similarity \(CLS↑\\uparrow\), the average pairwise cosine similarity between parallel samples across all language pairs, where higher values indicate more language\-agnostic representations; and \(2\)Silhouette Score \(SS↓\\downarrow\)\[[44](https://arxiv.org/html/2606.26466#bib.bib12)\], the standard cosine\-distance silhouette score measuring how strongly hidden states cluster by language, where lower values indicate weaker language separability\. We probe early, middle, and final layers, using layer 1, the 50%\-depth layer, and the last layer for each model\. Metrics are computed onN=50N\{=\}50parallel MGSM instances across the seven training languages\. For soft\-thinking representations, we first run a discrete\-token forward pass to construct soft tokens, then run a second pass with these soft tokens as inputs to extract the corresponding hidden states\. Full results for models across the Qwen3, Gemma3, and DeepSeek\-R1\-Distill\[[9](https://arxiv.org/html/2606.26466#bib.bib32)\]families are in Appendix[F\.2](https://arxiv.org/html/2606.26466#A6.SS2)\.

#### Cross\-lingual similarity follows an inverted\-U \(low\-high\-low\) pattern across early, middle, and final layers\.

![Refer to caption](https://arxiv.org/html/2606.26466v1/x3.png)Figure 3:Layer\-wise CLS and SS for Gemma3\-12B on MGSM under discrete and soft thinking\. CLS peaks in the middle layer and drops at the final layer, where soft thinking yields the largest gains\.Table[3](https://arxiv.org/html/2606.26466#S4.T3)compares representations under discrete and inference\-time soft thinking on MGSM for three base models, isolating the effect of soft tokens at inference time before introducing SOLAR’s training\-time alignment\. With discrete tokens, CLS is high in early and middle layers but drops sharply at the final layer\. Figure[3](https://arxiv.org/html/2606.26466#S4.F3)visualizes this trend across all layers for Gemma3\-12B\. SS shows a U\-shaped pattern, reaching its lowest values at the middle layer before rising sharply at the final layer\. This pattern matches prior observations that LLMs share a more language\-agnostic semantic space mid\-network while final layers reintroduce language\-specific structure\[[27](https://arxiv.org/html/2606.26466#bib.bib81),[57](https://arxiv.org/html/2606.26466#bib.bib4),[66](https://arxiv.org/html/2606.26466#bib.bib73),[55](https://arxiv.org/html/2606.26466#bib.bib72),[37](https://arxiv.org/html/2606.26466#bib.bib74)\]\.

#### Soft tokens improve final\-layer alignment but do not dissolve language clusters\.

Compared to discrete thinking, soft tokens raise final\-layer CLS across all models, with the largest gain in Gemma3\-12B \(Δ\\DeltaCLS=\+0\.68, Table[3](https://arxiv.org/html/2606.26466#S4.T3)\)\. Early and middle layers see little benefit and can even degrade \(e\.g\.,Qwen3\-8B hasΔ\\DeltaCLS=−\-0\.03 at the middle layer\)\. These layers already reach high CLS under discrete thinking, leaving little room to improve, and the distributional shift from soft tokens may slightly disrupt their alignment\. SS improvements are limited and inconsistent: only Gemma3\-12B shows a large decrease \(Δ\\DeltaSS=−\-0\.73\), while other models show negligible change or slight increases\. Language clusters thus largely remain intact for most models \(Figure[2](https://arxiv.org/html/2606.26466#S4.F2)\)\.

#### SOLAR produces the strongest cross\-lingual representations\.

Having shown that soft tokens carry useful cross\-lingual signal but that inference\-time use alone does not improve performance \(§[3\.2](https://arxiv.org/html/2606.26466#S3.SS2)\), we examine whether fine\-tuning reshapes representations more effectively \(Table[4](https://arxiv.org/html/2606.26466#S4.T4); same 50 MGSM samples\)\. The base Qwen3\-4B shows the inverted\-U pattern, with CLS dropping to 0\.14 at the final layer\. SFT raises final\-layer CLS to 0\.30 \(\+0\.16\), while SOLAR reaches 0\.50 \(\+0\.36, more than 2×\\timesthe SFT gain\) and substantially reduces final\-layer SS from 0\.89 to 0\.77 \(−\-0\.12\), indicating weaker language clustering\. Applying soft\-thinking inference to fine\-tuned models improves SFT’s CLS \(0\.30→\\rightarrow0\.40\) but slightly reduces SOLAR’s \(0\.50→\\rightarrow0\.42\)\. This suggests that SOLAR already internalizes the alignment benefit that soft tokens provide, and that training\-time alignment subsumes what inference\-time soft thinking can recover post\-hoc\. On Qwen3\-8B, SOLAR’s discrete\-thinking gain is larger \(Δ\\DeltaCLS = \+0\.53\) and, unlike on 4B, remains nearly intact under soft thinking \(\+0\.51\)\.

Table 3:Cross\-lingual representation alignment on MGSM under discrete \(Disc\-\) vs\. soft \(Soft\-\) thinking\.Δ\\Deltacolumns are Soft−\-Disc\. Mean overN=50N\{=\}50parallel samples\.Qwen3\-8BGemma3\-12BDeepSeek\-R1\-Distill\-Qwen\-7BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS0\.820\.970\.230\.991\.000\.250\.741\.000\.24Soft\-CLS0\.830\.940\.360\.931\.000\.930\.730\.970\.25Δ\\DeltaCLS\+0\.01\-0\.03\+0\.13\-0\.06\+0\.00\+0\.68\-0\.01\-0\.03\+0\.01Disc\-SS0\.530\.150\.870\.310\.210\.820\.550\.110\.83Soft\-SS0\.600\.270\.830\.540\.080\.090\.590\.200\.82Δ\\DeltaSS\+0\.07\+0\.12\-0\.04\+0\.23\-0\.13\-0\.73\+0\.04\+0\.09\-0\.01Table 4:Final\-layer cross\-lingual representation analysis on Qwen3\-4B/8B and their fine\-tuned variants\.Δ\\Deltarows are training effects\. Results for early/middle layers in Appendix[F\.2](https://arxiv.org/html/2606.26466#A6.SS2)\.Discrete ThinkingSoft ThinkingCLSSSCLSSSQwen3\-4B0\.140\.890\.240\.88\+ SFT0\.300\.860\.400\.85Δ\\Delta\+0\.16\-0\.03\+0\.16\-0\.03\+ SOLAR0\.500\.770\.420\.81Δ\\Delta\+0\.36\-0\.12\+0\.18\-0\.07Qwen3\-8B0\.230\.870\.360\.83\+ SFT0\.500\.860\.560\.86Δ\\Delta\+0\.27\-0\.01\+0\.20\+0\.03\+ SOLAR0\.760\.770\.870\.76Δ\\Delta\+0\.53\-0\.10\+0\.51\-0\.07

Table 5:Distribution\-level target\-script ratio \(%\) on MGSM under top\-kk/top\-pp\.ModelZhJaThTeAvg\.\(a\) Top\-kk\(k=30k\{=\}30\)Qwen3\-4B82\.2678\.8787\.7065\.3778\.55\+ SFT91\.2992\.2691\.9165\.2185\.17\+ SOLAR79\.3481\.1491\.2198\.3787\.51Qwen3\-8B92\.4919\.7327\.9911\.9738\.05\+ SFT93\.0993\.0994\.4169\.0587\.41\+ SOLAR82\.8377\.7991\.7598\.9987\.84\(b\) Top\-pp\(p=0\.9p\{=\}0\.9\)Qwen3\-4B94\.5795\.8497\.2294\.3795\.50\+ SFT95\.6997\.3598\.4198\.7597\.55\+ SOLAR95\.7597\.1698\.4699\.3497\.68Qwen3\-8B97\.9921\.7532\.5913\.1541\.37\+ SFT96\.2197\.1498\.4798\.7897\.65\+ SOLAR95\.6896\.0597\.3899\.4797\.14

### 4\.2Behavioral analysis

We examine whether fine\-tuning changes the model’s reasoning behavior on MGSM \(Zh, Ja, Th, Te\) at thetokenlevel \(the generated tokens of each reasoning trace\) and thedistributionlevel \(the top\-kk/top\-ppcandidates in the output distribution during teacher\-forced decoding\)\. At both levels, every token is classified into one of three categories using Unicode code\-point ranges:target\-script\(the prompt language’s native writing system,e\.g\.,CJK ideographs for Chinese\),Latin\(Unicode names containinglatin, used as a proxy for English\), andneutral\(digits, punctuation, math symbols, and whitespace\)\. We define the*target\-script ratio*as\#​target/\(\#​target\+\#​Latin\)\\\#\\,\\text\{target\}\\,/\\,\(\\\#\\,\\text\{target\}\+\\\#\\,\\text\{Latin\}\)to exclude neutral tokens so that script preference is not diluted by language\-agnostic symbols\.

#### Fine\-tuning preserves native language choice at the token level\.

Qwen3\-4B reasons almost entirely in the target script \(98\.81%\)\. SFT and SOLAR maintain comparable ratios \(Figure[4](https://arxiv.org/html/2606.26466#S4.F4)\)\. The ratio is nearly identical for correct and incorrect answers \(averageΔ<0\.5%\\Delta<0\.5\\%; Table[32](https://arxiv.org/html/2606.26466#A6.T32)\), indicating that fine\-tuning improves task performance without shifting language choice toward English\.

#### SFT and SOLAR achieve high target\-script ratios through distinct mechanisms\.

![Refer to caption](https://arxiv.org/html/2606.26466v1/x4.png)Figure 4:Token\-level target\-script ratio \(%\) on MGSM of Qwen3\-4B and its fine\-tuned variants\. Ratios remain high with little variation across methods, suggesting that performance differences are not driven by changes in target\-script usage\.The 8B base model shows a stronger English preference than 4B \(e\.g\.,Telugu 11\.97% vs\. above 65%; Figure[5](https://arxiv.org/html/2606.26466#S4.F5), Table[5](https://arxiv.org/html/2606.26466#S4.T5)\)\. SFT and SOLAR close this gap \(8B average 38\.05%→\\rightarrowabove 87%\) through different mechanisms: SFT raises target\-script token density in the top\-30 predictions, while SOLAR suppresses Latin competition\. The contrast is clearest for Telugu, where SOLAR nearly eliminates Latin competition \(98\.37% for 4B\) but SFT leaves it largely unchanged \(65\.21% vs\. 65\.37%\)\. Under top\-ppsampling, all fine\-tuned models reach 95–99% target\-script ratios, so remaining cross\-lingual competition is confined to low\-probability alternatives\.

![Refer to caption](https://arxiv.org/html/2606.26466v1/x5.png)Figure 5:Distribution\-level token composition by language script on MGSM for Qwen3\-4B, Qwen3\-8B, and their training variants under top\-kk\(k=30k\{=\}30, top row\) and top\-pp\(p=0\.9p\{=\}0\.9, bottom row\)\. Each bar decomposes tokens into target\-script \(dark teal\), Latin \(medium green\), and neutral \(light green\) categories\. SFT increases the density of target\-script tokens in the top predictions, while SOLAR primarily suppresses Latin competition, most notably for low\-resource scripts such as Telugu\.

### 4\.3Hyperparameter sensitivity

#### Alignment weightλ\\lambda\.

Table 6:Hyperparameter ablation \(λ\\lambda,τ\\tau\) for SOLAR with Qwen3\-4B on MGSM\.ConfigurationAccuracyCLC\(a\) Alignment weightλ\\lambda0\.2578\.6175\.730\.5078\.2775\.610\.7575\.8373\.731\.0069\.2563\.67\(b\) Soft\-token temperatureτ\\tau0\.660\.4857\.031\.078\.2775\.612\.076\.4173\.13We vary the alignment weightλ∈\{0\.25,0\.50,0\.75,1\.00\}\\lambda\\in\\\{0\.25,0\.50,0\.75,1\.00\\\}, with the soft\-token temperatureτ=1\.0\\tau\{=\}1\.0fixed\. Bothλ=0\.25\\lambda\{=\}0\.25andλ=0\.50\\lambda\{=\}0\.50perform strongly on MGSM \(Table[6](https://arxiv.org/html/2606.26466#S4.T6)\), andλ=0\.50\\lambda\{=\}0\.50also leads on the harder benchmarks \(Table[16](https://arxiv.org/html/2606.26466#A6.T16)\)\. Larger weights degrade performance, withλ=1\.00\\lambda\{=\}1\.00falling below the base model, suggesting that excessive alignment overwhelms the cross\-entropy objective\. We adoptλ=0\.50\\lambda\{=\}0\.50for all main experiments in §[3\.2](https://arxiv.org/html/2606.26466#S3.SS2)\.

#### Soft\-token temperatureτ\\tau\.

We vary the soft\-token temperatureτ∈\{0\.6,1\.0,2\.0\}\\tau\\in\\\{0\.6,1\.0,2\.0\\\}, withλ=0\.50\\lambda\{=\}0\.50fixed\.τ\\tausets the sharpness of the soft\-token distribution: lowerτ\\tauyields more peaked distributions near discrete tokens, while higherτ\\tauyields smoother mixtures over the vocabulary\. Settingτ=1\.0\\tau\{=\}1\.0performs best \(78\.3% accuracy, 75\.6% CLC; Table[6](https://arxiv.org/html/2606.26466#S4.T6)\)\. Lowering it toτ=0\.6\\tau\{=\}0\.6degrades sharply \(60\.5% accuracy, 57\.0% CLC\), indicating that overly peaked distributions lose the cross\-lingual signal, whereasτ=2\.0\\tau\{=\}2\.0yields only a moderate drop \(76\.4% accuracy, 73\.1% CLC\)\.

### 4\.4What drives the alignment gain?

To isolate which component of SOLAR’s objective drives its gain, we vary one design choice at a time while holding the rest of the pipeline fixed\. The five choices we study are the alignment target, the pooling granularity, the top\-kkfilter, the alignment representation space, and the pivot language\.

#### Alignment target\.

We compare four alignment targets for the auxiliary loss: \(1\)Discrete, the straight\-through embedding of the top\-1 token \(𝐞t=𝐄​\[yt\]\\mathbf\{e\}\_\{t\}=\\mathbf\{E\}\[y\_\{t\}\]\), replacing the soft\-token embedding with one hard token; \(2\)Final\-hidden, the mean\-pooled final\-layer hidden state, testing whether any final\-layer matching signal suffices; \(3\)Untruncated, the soft\-token embedding over the full vocabulary with no top\-kkfilter; and \(4\)SOLAR, the default top\-k=30k\{=\}30soft\-token embedding\. On Qwen3\-4B, SOLAR is the best alignment target \(48\.25% average accuracy, 41\.51% CLC; Table[7](https://arxiv.org/html/2606.26466#S4.T7)\)\. The other targets cluster within run\-to\-run noise: Discrete and Final\-hidden stay at the SFT level \(46\.00% and 46\.09% vs\. 45\.67%\), and Untruncated, the strongest alternative \(46\.55%\), still trails SOLAR\. A single hard token \(Discrete\) thus recovers almost none of SOLAR’s gain, so top\-kktruncation carries semantic value beyond compute savings\. We attribute this to two factors\. First, the long tail kept by Untruncated injects cross\-lingual noise that top\-kkfiltering removes\. Second, aligning only the final hidden state \(Final\-hidden\) underperforms because the embedding\-space soft\-token mixture is more language\-invariant than the final hidden state itself\.

Table 7:Alignment\-target ablation for SOLAR with Qwen3\-4B\.MGSMAIME 2024AIME 2025GPQAAvg\.Alignment targetAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCQwen3\-4B\+\+SFT76\.2572\.2934\.7625\.1727\.9020\.4443\.7533\.1545\.6737\.76\+ Discrete76\.3773\.5035\.6326\.0628\.1221\.4143\.8633\.4746\.0038\.61\+ Final\-hidden76\.9773\.7935\.4725\.9127\.8720\.0544\.0534\.5146\.0938\.57\+ Untruncated77\.1974\.7237\.0327\.6327\.9322\.9344\.0434\.4646\.5539\.94\+ SOLAR \(k=30k\{=\}30\)78\.2775\.6138\.5229\.6730\.9524\.5145\.2436\.2648\.2541\.51

#### Pooling granularity\.

We fix the soft\-token target and vary the pooling granularity of the alignment loss: \(1\)Answer\-only, pooling over the final\\boxed\{\}span; \(2\)Step\-level, pooling within each step \(split on\\n\\n\) and matched by relative position; and \(3\)Full, the default single pooled pair over the entire response\. Answer\-only pooling matches SFT \(MGSM 76\.00% vs\. 76\.25%, Table[8](https://arxiv.org/html/2606.26466#S4.T8)\), indicating that the alignment signal lives in the reasoning trace\. Step\-level, the finest variant with one pair per step, improves over Answer\-only but still trails Full \(MGSM 77\.04% vs\. 78\.27%\)\. We hypothesize that finer granularity helps only when steps correspond across languages\. Matching by relative position presumes a parallel step structure that the\\n\\nsplit does not guarantee, since equivalent reasoning often spans a different number of steps across languages\. Full\-response pooling avoids this misalignment\. Recovering the benefit of finer granularity likely requires content\-aware segmentation that aligns steps across languages, which we leave to future work\.

Table 8:Pooling\-granularity ablation for SOLAR with Qwen3\-4B\.MGSMAIME 2024AIME 2025GPQAAvg\.Pooling granularityAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCQwen3\-4B\+\+SFT76\.2572\.2934\.7625\.1727\.9020\.4443\.7533\.1545\.6737\.76\+ Answer\-only76\.0071\.9134\.9325\.9227\.9220\.9444\.9534\.9645\.9538\.43\+ Step\-level77\.0473\.3437\.0328\.9529\.0123\.8846\.1537\.8147\.3141\.00\+ Full \(SOLAR\)78\.2775\.6138\.5229\.6730\.9524\.5145\.2436\.2648\.2541\.51

#### Top\-kksensitivity\.

We sweepk∈\{1,5,10,30,50\}k\\in\\\{1,5,10,30,50\\\}for the top\-kkfilter defining the soft expectation \(Table[9](https://arxiv.org/html/2606.26466#S4.T9)\)\.k=1k\{=\}1recovers the Discrete target, since a top\-1 filter with renormalization is a hard selection\. Accuracy rises steeply fromk=1k\{=\}1tok=10k\{=\}10, then plateaus:k∈\{10,30,50\}k\\in\\\{10,30,50\\\}all reach roughly 48% average accuracy, with no value dominating across benchmarks\. Becausekkaffects only the alignment\-loss sum over the top\-kkdistribution, largerkkraises per\-step compute with no accuracy gain\. We adoptk=30k\{=\}30, which attains the best average accuracy among the plateau values\.

Table 9:Top\-kkablation for SOLAR with Qwen3\-4B\.MGSMAIME 2024AIME 2025GPQAAvg\.Top\-kkAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCSFT76\.2572\.2934\.7625\.1727\.9020\.4443\.7533\.1545\.6737\.76k=1k\{=\}1\(Discrete\)76\.3773\.5035\.6326\.0628\.1221\.4143\.8633\.4746\.0038\.61k=5k\{=\}577\.0073\.9236\.4626\.9728\.6422\.2044\.8935\.9646\.7539\.76k=10k\{=\}1078\.3676\.0238\.4029\.1230\.9324\.4244\.9235\.9848\.1541\.19k=30k\{=\}30\(SOLAR\)78\.2775\.6138\.5229\.6730\.9524\.5145\.2436\.2648\.2541\.51k=50k\{=\}5078\.0875\.1138\.3429\.0031\.1024\.8945\.2936\.5548\.2041\.39

#### Alignment representation space\.

Table 10:Representation\-space and pivot\-language ablation for SOLAR with Qwen3\-4B on MGSM\.ConfigurationAccuracyCLC\(a\) Representation spaceℝd\\mathbb\{R\}^\{d\}\(emb\.\)78\.2775\.61ℝ\|V\|\\mathbb\{R\}^\{\|V\|\}\(prob\.\)60\.2744\.76\(b\) Pivot languageℓ∗\\ell^\{\*\}English \(En\)78\.2775\.61Chinese \(Zh\)74\.5070\.97French \(Fr\)72\.8766\.67We vary the alignment representation space betweenℝd\\mathbb\{R\}^\{d\}\(soft\-token embeddings\) andℝ\|V\|\\mathbb\{R\}^\{\|V\|\}\(probability distributions\), withλ=0\.50\\lambda\{=\}0\.50andτ=1\.0\\tau\{=\}1\.0fixed\. Aligning in probability space degrades accuracy to 60\.3% and CLC to 44\.8% \(Table[10](https://arxiv.org/html/2606.26466#S4.T10)\)\. This degradation suggests that token distributions have largely disjoint support across languages, making cosine distance less informative\. Projecting them into the shared embedding spaceℝd\\mathbb\{R\}^\{d\}instead preserves cross\-lingual correspondences and provides a better basis for alignment\.

#### Pivot languageℓ∗\\ell^\{\*\}\.

We vary the pivot languageℓ∗\\ell^\{\*\}across English, Chinese, and French, withλ=0\.50\\lambda\{=\}0\.50andτ=1\.0\\tau\{=\}1\.0fixed\. English gives the best improvement, followed by Chinese \(Table[10](https://arxiv.org/html/2606.26466#S4.T10)\)\. The gap concentrates in low\-resource languages: MGSM Swahili accuracy drops from 53\.48% \(En\) to 40\.56% \(Zh\) and 36\.80% \(Fr\) \(Table[17](https://arxiv.org/html/2606.26466#A6.T17)\), showing that transfer effectiveness depends on the pivot’s pre\-trained representation strength, with English as the strongest anchor\.

## 5Related work

#### Multilingual reasoning\.

LLMs trained on multilingual corpora exhibit inconsistent cross\-lingual reasoning performance\[[48](https://arxiv.org/html/2606.26466#bib.bib10),[39](https://arxiv.org/html/2606.26466#bib.bib6)\], and recent work attributes these disparities to internal mechanisms\.Wanget al\.\[[57](https://arxiv.org/html/2606.26466#bib.bib4)\]show that errors arise in the final\-layer transition from language\-agnostic intermediate representations to language\-specific outputs\.Liuet al\.\[[31](https://arxiv.org/html/2606.26466#bib.bib3)\]identify language\-specific attention heads that mediate cross\-lingual attention transfer\.Limet al\.\[[27](https://arxiv.org/html/2606.26466#bib.bib81)\]find that larger models rely more on language\-specific representations\. Methods for improving multilingual reasoning, particularly for low\-resource languages, fall broadly into training\-based and inference\-based categories\[[14](https://arxiv.org/html/2606.26466#bib.bib43)\]\. Training\-based methods leverage parallel multilingual data during pre\-training\[[60](https://arxiv.org/html/2606.26466#bib.bib70)\]or post\-training\[[69](https://arxiv.org/html/2606.26466#bib.bib58)\], language\-mixed CoT data\[[50](https://arxiv.org/html/2606.26466#bib.bib68)\], adapter merging\[[76](https://arxiv.org/html/2606.26466#bib.bib64)\], and representation alignment\[[2](https://arxiv.org/html/2606.26466#bib.bib7),[3](https://arxiv.org/html/2606.26466#bib.bib57)\]\. A recent wave of multilingual reinforcement learning \(RL\) methods uses English as a pivot with translation\-quality, language\-consistency, or self\-feedback rewards\[[74](https://arxiv.org/html/2606.26466#bib.bib15),[20](https://arxiv.org/html/2606.26466#bib.bib16),[72](https://arxiv.org/html/2606.26466#bib.bib33)\]\. Inference\-based methods include in\-context learning\[[52](https://arxiv.org/html/2606.26466#bib.bib66)\], activation steering\[[37](https://arxiv.org/html/2606.26466#bib.bib74)\], test\-time scaling\[[70](https://arxiv.org/html/2606.26466#bib.bib69)\], and inference\-time linear transformations between source\- and target\-language hidden states\[[58](https://arxiv.org/html/2606.26466#bib.bib14)\]\.

#### Continuous\-space reasoning\.

While CoT prompting\[[61](https://arxiv.org/html/2606.26466#bib.bib41)\]enhances LLM reasoning through discrete tokens, a growing line of work argues that reasoning may instead unfold in continuous representations that encode richer information than discrete tokens\[[6](https://arxiv.org/html/2606.26466#bib.bib8),[51](https://arxiv.org/html/2606.26466#bib.bib28)\]\. Several methods train LLMs to reason directly in continuous space\[[65](https://arxiv.org/html/2606.26466#bib.bib29),[47](https://arxiv.org/html/2606.26466#bib.bib9)\]; for instance, COCONUT\[[18](https://arxiv.org/html/2606.26466#bib.bib39)\]feeds the last hidden state back as the next input embedding, allowing multiple reasoning steps to unfold in parallel\.Buttet al\.\[[4](https://arxiv.org/html/2606.26466#bib.bib17)\]train continuous CoTs with reinforcement learning by injecting noise into the input embedding mixture for exploration, and report that decoding with discrete tokens at inference on top of a continuous\-CoT trained model performs best\. Among training\-free approaches, Soft Thinking\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\]replaces discrete token embeddings with probability\-weighted soft tokens for richer inference\-time representations, and theoretical work further shows that continuous CoT can efficiently solve graph\-reachability problems\[[79](https://arxiv.org/html/2606.26466#bib.bib21)\]\. Taken together, these results suggest that continuous\-space reasoning carries expressive advantages over its discrete counterpart\[[51](https://arxiv.org/html/2606.26466#bib.bib28)\]\.

## 6Conclusion

We introduce SOLAR, an auxiliary training objective that aligns soft\-token representations across parallel multilingual reasoning traces\. SOLAR improves task accuracy and cross\-lingual consistency over standard SFT across four multilingual benchmarks, with the largest gains on low\-resource languages\. At the representation level, SOLAR more than doubles SFT’s final\-layer cross\-lingual similarity gain and substantially reduces language\-cluster separability, showing that aligning soft\-token representations helps preserve shared semantics and improve cross\-lingual consistency\.

## Acknowledgments

This research is supported in part by the NSF under grant numbers IIS\-2144493, IIS\-2052498, SMA\-2418946, and NAIRR250217, in addition to a gift from Google\. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the author\(s\) and do not necessarily reflect the views of the National Science Foundation\.

## References

- \[1\]J\. Barua, S\. Eisape, K\. Yin, and A\. Suhr\(2026\)Long chain\-of\-thought reasoning across languages\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=2kKXbsRhYI)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px3.p1.1),[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px1.p1.2)\.
- \[2\]M\. Bu, S\. Zhang, Z\. He, H\. Wu, and Y\. Feng\(2025\-11\)AlignX: advancing multilingual large language models with multilingual representation alignment\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 6460–6489\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.328/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.328),ISBN 979\-8\-89176\-332\-6Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§D\.5](https://arxiv.org/html/2606.26466#A4.SS5.SSS0.Px4.p1.3),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px4.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[3\]Y\. Bu, X\. Liu, Z\. Ren, Y\. Yang, and J\. Dai\(2026\)Align once, benefit multilingually: enforcing multilingual consistency for LLM safety alignment\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ueknOG1wXL)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[4\]N\. Butt, A\. Kwiatkowski, I\. Labiad, J\. Kempe, and Y\. Ollivier\(2025\)Soft tokens, hard truths\.External Links:2509\.19170,[Link](https://arxiv.org/abs/2509.19170)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[5\]K\. Chen, M\. Zhang, and Y\. Cao\(2025\)Less data less tokens: multilingual unification learning for efficient test\-time reasoning in llms\.External Links:2506\.18341,[Link](https://arxiv.org/abs/2506.18341)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[6\]X\. Chen, A\. Zhao, H\. Xia, X\. Lu, H\. Wang, Y\. Chen, W\. Zhang, J\. Wang, W\. Li, and X\. Shen\(2025\)Reasoning beyond language: a comprehensive survey on latent chain\-of\-thought reasoning\.External Links:2505\.16782,[Link](https://arxiv.org/abs/2505.16782)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[7\]Y\. Chen, V\. Shah, and A\. Ritter\(2025\-07\)Translation and fusion improves cross\-lingual information extraction\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 7744–7764\.External Links:[Link](https://aclanthology.org/2025.acl-long.382/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.382),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[8\]K\. Cobbe, V\. Kosaraju, M\. Bavarian, M\. Chen, H\. Jun, L\. Kaiser, M\. Plappert, J\. Tworek, J\. Hilton, R\. Nakano, C\. Hesse, and J\. Schulman\(2021\)Training verifiers to solve math word problems\.External Links:2110\.14168,[Link](https://arxiv.org/abs/2110.14168)Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px2.p1.1)\.
- \[9\]DeepSeek\-AI\(2025\)DeepSeek\-r1: incentivizing reasoning capability in llms via reinforcement learning\.External Links:2501\.12948,[Link](https://arxiv.org/abs/2501.12948)Cited by:[§D\.1](https://arxiv.org/html/2606.26466#A4.SS1.p1.1),[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.p1.10)\.
- \[10\]F\. Faisal, K\. Song, S\. Wang, S\. Ma, S\. Liu, H\. Deng, and S\. R\. Indurthi\(2025\)Aligning multilingual reasoning with verifiable semantics from a high\-resource expert model\.External Links:2509\.25543,[Link](https://arxiv.org/abs/2509.25543)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[11\]N\. Foroutan, P\. Teiletche, A\. K\. Tarun, and A\. Bosselut\(2025\)Revisiting multilingual data mixtures in language model pretraining\.External Links:2510\.25947,[Link](https://arxiv.org/abs/2510.25947)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[12\]L\. C\. L\. Gamboa, Y\. Feng, and M\. G\. Lee\(2025\-11\)Social bias in multilingual language models: a survey\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 27857–27880\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1416/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1416),ISBN 979\-8\-89176\-332\-6Cited by:[Appendix B](https://arxiv.org/html/2606.26466#A2.p3.1)\.
- \[13\]S\. Gao, F\. Nan, Y\. Zhang, Y\. Huang, K\. Tan, and Z\. Yu\(2025\-04\)A mixed\-language multi\-document news summarization dataset and a graphs\-based extract\-generate model\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 9255–9265\.External Links:[Link](https://aclanthology.org/2025.naacl-long.468/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.468),ISBN 979\-8\-89176\-189\-6Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px2.p1.1)\.
- \[14\]A\. Ghosh, D\. Datta, S\. Saha, and C\. Agarwal\(2025\-11\)A survey of multilingual reasoning in language models\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 8920–8936\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.474/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.474),ISBN 979\-8\-89176\-335\-7Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[15\]Google DeepMind\(2025\)Gemini 2\.0 Flash Model Card\.External Links:[Link](https://storage.googleapis.com/model-cards/documents/gemini-2-flash.pdf)Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px1.p1.2)\.
- \[16\]H\. A\. Gozeten, M\. E\. Ildiz, X\. Zhang, H\. Harutyunyan, A\. S\. Rawat, and S\. Oymak\(2025\)Continuous chain of thought enables parallel exploration and reasoning\.InICML 2025 Workshop on Methods and Opportunities at Small Scale,External Links:[Link](https://openreview.net/forum?id=1ORJaYuMJc)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1)\.
- \[17\]P\. Guo, Y\. Ren, Y\. Hu, Y\. Cao, Y\. Li, and H\. Huang\(2024\)Steering large language models for cross\-lingual information retrieval\.InProceedings of the 47th International ACM SIGIR Conference on Research and Development in Information Retrieval,SIGIR ’24,New York, NY, USA,pp\. 585–596\.External Links:ISBN 9798400704314,[Link](https://doi.org/10.1145/3626772.3657819),[Document](https://dx.doi.org/10.1145/3626772.3657819)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[18\]S\. Hao, S\. Sukhbaatar, D\. Su, X\. Li, Z\. Hu, J\. E\. Weston, and Y\. Tian\(2025\)Training large language models to reason in a continuous latent space\.InWorkshop on Reasoning and Planning for Large Language Models,External Links:[Link](https://openreview.net/forum?id=KrWSrrYGpT)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[19\]S\. Huang, Y\. Ding, J\. Pan, and Y\. Zhang\(2026\)Beyond english\-centric training: how reinforcement learning improves cross\-lingual reasoning in LLMs\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=hdrG6SaTcA)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px1.p1.1),[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[20\]X\. Huang, Z\. Lai, Z\. Huang, J\. Chen, and S\. Huang\(2026\)TAPO: translation augmented policy optimization for multilingual mathematical reasoning\.External Links:2603\.25419,[Link](https://arxiv.org/abs/2603.25419)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px1.p1.1),[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[21\]J\. Hwang, K\. Tanmay, S\. Lee, A\. Agrawal, H\. Palangi, K\. Ayush, I\. Fiete, and P\. P\. Liang\(2025\)Learn globally, speak locally: bridging the gaps in multilingual reasoning\.External Links:2507\.05418,[Link](https://arxiv.org/abs/2507.05418)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[22\]Kaggle\(2025\)AIME\-2025 Dataset\.Kaggle\.External Links:[Link](https://www.kaggle.com/datasets/hengck23/hengck23-aime-2025)Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px2.p1.3)\.
- \[23\]T\. Kew, F\. Schottmann, and R\. Sennrich\(2024\-11\)Turning English\-centric LLMs into polyglots: how much multilinguality is needed?\.InFindings of the Association for Computational Linguistics: EMNLP 2024,Y\. Al\-Onaizan, M\. Bansal, and Y\. Chen \(Eds\.\),Miami, Florida, USA,pp\. 13097–13124\.External Links:[Link](https://aclanthology.org/2024.findings-emnlp.766/),[Document](https://dx.doi.org/10.18653/v1/2024.findings-emnlp.766)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[24\]H\. Lai and M\. Nissim\(2024\-08\)MCoT: multilingual instruction tuning for reasoning consistency in language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 12012–12026\.External Links:[Link](https://aclanthology.org/2024.acl-long.649/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.649)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[25\]J\. Lee, S\. Hong, H\. Moon, and H\. Lim\(2025\-07\)Cross\-lingual optimization for language transfer in large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 15100–15119\.External Links:[Link](https://aclanthology.org/2025.acl-long.734/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.734),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[26\]Z\. Li, R\. Xu, X\. Wang, J\. Liu, Z\. Chen, Q\. Mao, J\. Yang, H\. Guo, L\. Xiao, and M\. LI\(2025\)Steering merged LLMs for multilingual reasoning with coefficient optimization\.External Links:[Link](https://openreview.net/forum?id=TQjGGTnotA)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[27\]Z\. W\. Lim, A\. F\. Aji, and T\. Cohn\(2025\)Language\-specific latent process hinders cross\-lingual performance\.arXiv preprint arXiv:2505\.13141\.External Links:[Link](https://arxiv.org/abs/2505.13141)Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px3.p1.9),[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[28\]H\. Lin, Y\. Zhao, W\. Han, P\. Guo, BINBINLIU, Y\. Zhang, B\. Zhang, T\. Wang, and Y\. Zheng\(2025\)From translation to multilinguality: revisit the role of parallel data in multilingual LLM pretraining\.External Links:[Link](https://openreview.net/forum?id=1gbJ8euERb)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[29\]D\. Liu and J\. Niehues\(2025\-07\)Middle\-layer representation alignment for cross\-lingual transfer in fine\-tuned LLMs\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 15979–15996\.External Links:[Link](https://aclanthology.org/2025.acl-long.778/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.778),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§D\.5](https://arxiv.org/html/2606.26466#A4.SS5.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px4.p1.1)\.
- \[30\]T\. Liu, J\. Qi, M\. Sachan, R\. Cotterell, R\. Fernández, and A\. Bisazza\(2026\)Optimizing language models for crosslingual knowledge consistency\.External Links:[Link](https://openreview.net/forum?id=nupbCYlmEE)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[31\]X\. Liu, Q\. Song, Q\. Zhou, H\. Du, S\. Xu, W\. Jiang, W\. Zhang, and X\. Jia\(2026\)Focusing on language: revealing and exploiting language attention heads in multilingual large language models\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 32195–32203\.Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[32\]I\. Loshchilov and F\. Hutter\(2019\)Decoupled weight decay regularization\.External Links:1711\.05101,[Link](https://arxiv.org/abs/1711.05101)Cited by:[§D\.2](https://arxiv.org/html/2606.26466#A4.SS2.p1.7)\.
- \[33\]N\. Muennighoff, Z\. Yang, W\. Shi, X\. L\. Li, L\. Fei\-Fei, H\. Hajishirzi, L\. Zettlemoyer, P\. Liang, E\. Candes, and T\. Hashimoto\(2025\-11\)S1: simple test\-time scaling\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 20275–20321\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1025/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1025),ISBN 979\-8\-89176\-332\-6Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px1.p1.2)\.
- \[34\]C\. Nimo, T\. Olatunji, A\. T\. Owodunni, T\. Abdullahi, E\. Ayodele, M\. Sanni, E\. C\. Aka, F\. Omofoye, F\. Yuehgoh, T\. Faniran, B\. F\. P\. Dossou, M\. O\. Yekini, J\. Kemp, K\. A\. Heller, J\. C\. Omeke, C\. A\. Md, N\. A\. Etori, A\. Ndiaye, I\. Okoh, E\. D\. Ocansey, W\. Kinara, M\. L\. Best, I\. Essa, S\. E\. Moore, C\. Fourie, and M\. N\. Asiedu\(2025\-07\)AfriMed\-QA: a pan\-African, multi\-specialty, medical question\-answering benchmark dataset\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 1948–1973\.External Links:[Link](https://aclanthology.org/2025.acl-long.96/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.96),ISBN 979\-8\-89176\-251\-0Cited by:[Appendix B](https://arxiv.org/html/2606.26466#A2.p1.1)\.
- \[35\]Q\. Peng, G\. Hu, Y\. Chai, and A\. Søgaard\(2025\-11\)Debiasing multilingual LLMs in cross\-lingual latent space\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 22582–22593\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1149/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1149),ISBN 979\-8\-89176\-332\-6Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[36\]T\. Pham and C\. Ngo\(2025\)Multimodal chain of continuous thought for latent\-space reasoning in vision\-language models\.InNeurIPS 2025 Workshop on Efficient Reasoning,External Links:[Link](https://openreview.net/forum?id=PXFTV1O2WJ)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1)\.
- \[37\]R\. Pokharel, A\. Agrawal, and T\. Nagar\(2026\)Cross\-lingual activation steering for multilingual language models\.External Links:2601\.16390,[Link](https://arxiv.org/abs/2601.16390)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p1.1),[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[38\]J\. Qi, S\. Chen, Z\. Xiong, R\. Fernández, D\. Bitterman, and A\. Bisazza\(2025\-11\)When models reason in your language: controlling thinking language comes at the cost of accuracy\.InFindings of the Association for Computational Linguistics: EMNLP 2025,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 20279–20296\.External Links:[Link](https://aclanthology.org/2025.findings-emnlp.1103/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-emnlp.1103),ISBN 979\-8\-89176\-335\-7Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px2.p1.3)\.
- \[39\]J\. Qi, R\. Fernández, and A\. Bisazza\(2023\-12\)Cross\-lingual consistency of factual knowledge in multilingual language models\.InProceedings of the 2023 Conference on Empirical Methods in Natural Language Processing,H\. Bouamor, J\. Pino, and K\. Bali \(Eds\.\),Singapore,pp\. 10650–10666\.External Links:[Link](https://aclanthology.org/2023.emnlp-main.658/),[Document](https://dx.doi.org/10.18653/v1/2023.emnlp-main.658)Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[40\]M\. R\. Qorib, J\. Li, and H\. T\. Ng\(2025\-07\)Just go parallel: improving the multilingual capabilities of large language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 33411–33424\.External Links:[Link](https://aclanthology.org/2025.acl-long.1602/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1602),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[41\]R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn\(2023\)Direct preference optimization: your language model is secretly a reward model\.InAdvances in Neural Information Processing Systems,A\. Oh, T\. Naumann, A\. Globerson, K\. Saenko, M\. Hardt, and S\. Levine \(Eds\.\),Vol\.36,pp\. 53728–53741\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2023/file/a85b405ed65c6477a4fe8302b5e06ce7-Paper-Conference.pdf)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px1.p1.1),[§D\.5](https://arxiv.org/html/2606.26466#A4.SS5.SSS0.Px3.p1.5)\.
- \[42\]S\. Rajbhandari, J\. Rasley, O\. Ruwase, and Y\. He\(2020\)ZeRO: memory optimizations toward training trillion parameter models\.InSC20: International Conference for High Performance Computing, Networking, Storage and Analysis,Vol\.,pp\. 1–16\.External Links:[Document](https://dx.doi.org/10.1109/SC41405.2020.00024)Cited by:[§D\.2](https://arxiv.org/html/2606.26466#A4.SS2.p1.7)\.
- \[43\]D\. Rein, B\. L\. Hou, A\. C\. Stickland, J\. Petty, R\. Y\. Pang, J\. Dirani, J\. Michael, and S\. R\. Bowman\(2024\)GPQA: a graduate\-level google\-proof q&a benchmark\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=Ti67584b98)Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px2.p1.3)\.
- \[44\]P\. J\. Rousseeuw\(1987\)Silhouettes: a graphical aid to the interpretation and validation of cluster analysis\.Journal of Computational and Applied Mathematics20,pp\. 53–65\.External Links:ISSN 0377\-0427,[Document](https://dx.doi.org/https%3A//doi.org/10.1016/0377-0427%2887%2990125-7),[Link](https://www.sciencedirect.com/science/article/pii/0377042787901257)Cited by:[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.p1.10)\.
- \[45\]Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo\(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px1.p1.1)\.
- \[46\]S\. She, W\. Zou, S\. Huang, W\. Zhu, X\. Liu, X\. Geng, and J\. Chen\(2024\-08\)MAPO: advancing multilingual reasoning through multilingual\-alignment\-as\-preference optimization\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),L\. Ku, A\. Martins, and V\. Srikumar \(Eds\.\),Bangkok, Thailand,pp\. 10015–10027\.External Links:[Link](https://aclanthology.org/2024.acl-long.539/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-long.539)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§D\.5](https://arxiv.org/html/2606.26466#A4.SS5.SSS0.Px3.p1.5),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px4.p1.1)\.
- \[47\]Z\. Shen, H\. Yan, L\. Zhang, Z\. Hu, Y\. Du, and Y\. He\(2025\-11\)CODI: compressing chain\-of\-thought into continuous space via self\-distillation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 677–693\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.36/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.36),ISBN 979\-8\-89176\-332\-6Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[48\]F\. Shi, M\. Suzgun, M\. Freitag, X\. Wang, S\. Srivats, S\. Vosoughi, H\. W\. Chung, Y\. Tay, S\. Ruder, D\. Zhou, D\. Das, and J\. Wei\(2023\)Language models are multilingual chain\-of\-thought reasoners\.InThe Eleventh International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=fR3wGCk-IXp)Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px2.p1.3),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[49\]G\. Son, J\. Hong, H\. Ko, and J\. Thorne\(2025\-07\)Linguistic generalizability of test\-time scaling in mathematical reasoning\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 14333–14368\.External Links:[Link](https://aclanthology.org/2025.acl-long.699/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.699),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[50\]G\. Son, D\. Yang, H\. L\. Patel, A\. Agarwal, H\. Ko, C\. lim, S\. Panda, M\. Kim, N\. drolia, D\. Choi, K\. Lee, and Y\. Yu\(2026\)Pushing on multilingual reasoning models with language\-mixed chain\-of\-thought\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=ABc5y3741T)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[51\]W\. Tan, J\. Li, J\. Ju, Z\. Luo, R\. Song, and J\. Luan\(2025\)Think silently, think fast: dynamic latent compression of LLM reasoning chains\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=AQsko3PPUe)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[52\]E\. Tanwar, S\. Dutta, M\. Borthakur, and T\. Chakraborty\(2023\-07\)Multilingual LLMs are better cross\-lingual in\-context learners with alignment\.InProceedings of the 61st Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),A\. Rogers, J\. Boyd\-Graber, and N\. Okazaki \(Eds\.\),Toronto, Canada,pp\. 6292–6307\.External Links:[Link](https://aclanthology.org/2023.acl-long.346/),[Document](https://dx.doi.org/10.18653/v1/2023.acl-long.346)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[53\]G\. Team, A\. Kamath, J\. Ferret, S\. Pathak, N\. Vieillard, R\. Merhej, S\. Perrin, T\. Matejovicova, A\. Ramé, M\. Rivière, L\. Rouillard, T\. Mesnard, G\. Cideron, J\. Grill, S\. Ramos, E\. Yvinec, M\. Casbon, E\. Pot, I\. Penchev, G\. Liu, F\. Visin, K\. Kenealy, L\. Beyer, X\. Zhai, A\. Tsitsulin, R\. Busa\-Fekete, A\. Feng, N\. Sachdeva, B\. Coleman, Y\. Gao, B\. Mustafa, I\. Barr, E\. Parisotto, D\. Tian, M\. Eyal, C\. Cherry, J\. Peter, D\. Sinopalnikov, S\. Bhupatiraju, R\. Agarwal, M\. Kazemi, D\. Malkin, R\. Kumar, D\. Vilar, I\. Brusilovsky, J\. Luo, A\. Steiner, A\. Friesen, A\. Sharma, A\. Sharma, A\. M\. Gilady, A\. Goedeckemeyer, A\. Saade, A\. Feng, A\. Kolesnikov, A\. Bendebury, A\. Abdagic, A\. Vadi, A\. György, A\. S\. Pinto, A\. Das, A\. Bapna, A\. Miech, A\. Yang, A\. Paterson, A\. Shenoy, A\. Chakrabarti, B\. Piot, B\. Wu, B\. Shahriari, B\. Petrini, C\. Chen, C\. L\. Lan, C\. A\. Choquette\-Choo, C\. Carey, C\. Brick, D\. Deutsch, D\. Eisenbud, D\. Cattle, D\. Cheng, D\. Paparas, D\. S\. Sreepathihalli, D\. Reid, D\. Tran, D\. Zelle, E\. Noland, E\. Huizenga, E\. Kharitonov, F\. Liu, G\. Amirkhanyan, G\. Cameron, H\. Hashemi, H\. Klimczak\-Plucińska, H\. Singh, H\. Mehta, H\. T\. Lehri, H\. Hazimeh, I\. Ballantyne, I\. Szpektor, I\. Nardini, J\. Pouget\-Abadie, J\. Chan, J\. Stanton, J\. Wieting, J\. Lai, J\. Orbay, J\. Fernandez, J\. Newlan, J\. Ji, J\. Singh, K\. Black, K\. Yu, K\. Hui, K\. Vodrahalli, K\. Greff, L\. Qiu, M\. Valentine, M\. Coelho, M\. Ritter, M\. Hoffman, M\. Watson, M\. Chaturvedi, M\. Moynihan, M\. Ma, N\. Babar, N\. Noy, N\. Byrd, N\. Roy, N\. Momchev, N\. Chauhan, N\. Sachdeva, O\. Bunyan, P\. Botarda, P\. Caron, P\. K\. Rubenstein, P\. Culliton, P\. Schmid, P\. G\. Sessa, P\. Xu, P\. Stanczyk, P\. Tafti, R\. Shivanna, R\. Wu, R\. Pan, R\. Rokni, R\. Willoughby, R\. Vallu, R\. Mullins, S\. Jerome, S\. Smoot, S\. Girgin, S\. Iqbal, S\. Reddy, S\. Sheth, S\. Põder, S\. Bhatnagar, S\. R\. Panyam, S\. Eiger, S\. Zhang, T\. Liu, T\. Yacovone, T\. Liechty, U\. Kalra, U\. Evci, V\. Misra, V\. Roseberry, V\. Feinberg, V\. Kolesnikov, W\. Han, W\. Kwon, X\. Chen, Y\. Chow, Y\. Zhu, Z\. Wei, Z\. Egyed, V\. Cotruta, M\. Giang, P\. Kirk, A\. Rao, K\. Black, N\. Babar, J\. Lo, E\. Moreira, L\. G\. Martins, O\. Sanseviero, L\. Gonzalez, Z\. Gleicher, T\. Warkentin, V\. Mirrokni, E\. Senter, E\. Collins, J\. Barral, Z\. Ghahramani, R\. Hadsell, Y\. Matias, D\. Sculley, S\. Petrov, N\. Fiedel, N\. Shazeer, O\. Vinyals, J\. Dean, D\. Hassabis, K\. Kavukcuoglu, C\. Farabet, E\. Buchatskaya, J\. Alayrac, R\. Anil, Dmitry, Lepikhin, S\. Borgeaud, O\. Bachem, A\. Joulin, A\. Andreev, C\. Hardin, R\. Dadashi, and L\. Hussenot\(2025\)Gemma 3 technical report\.External Links:2503\.19786,[Link](https://arxiv.org/abs/2503.19786)Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§D\.1](https://arxiv.org/html/2606.26466#A4.SS1.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px1.p1.2)\.
- \[54\]N\. Team, M\. R\. Costa\-jussà, J\. Cross, O\. Çelebi, M\. Elbayad, K\. Heafield, K\. Heffernan, E\. Kalbassi, J\. Lam, D\. Licht, J\. Maillard, A\. Sun, S\. Wang, G\. Wenzek, A\. Youngblood, B\. Akula, L\. Barrault, G\. M\. Gonzalez, P\. Hansanti, J\. Hoffman, S\. Jarrett, K\. R\. Sadagopan, D\. Rowe, S\. Spruit, C\. Tran, P\. Andrews, N\. F\. Ayan, S\. Bhosale, S\. Edunov, A\. Fan, C\. Gao, V\. Goswami, F\. Guzmán, P\. Koehn, A\. Mourachko, C\. Ropers, S\. Saleem, H\. Schwenk, and J\. Wang\(2022\)No language left behind: scaling human\-centered machine translation\.External Links:2207\.04672,[Link](https://arxiv.org/abs/2207.04672)Cited by:[§D\.5](https://arxiv.org/html/2606.26466#A4.SS5.SSS0.Px3.p1.5)\.
- \[55\]H\. Tezuka and N\. Inoue\(2025\-11\)The transfer neurons hypothesis: an underlying mechanism for language latent space transitions in multilingual LLMs\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 31742–31792\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1618/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1618),ISBN 979\-8\-89176\-332\-6Cited by:[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.SSS0.Px1.p1.1)\.
- \[56\]H\. Veeraboina\(2024\)AIME problem set 1983\-2024\.Kaggle\.External Links:[Link](https://www.kaggle.com/datasets/hemishveeraboina/aime-problem-set-1983-2024)Cited by:[§D\.4](https://arxiv.org/html/2606.26466#A4.SS4.SSS0.Px2.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px2.p1.3)\.
- \[57\]M\. Wang, H\. Adel, L\. Lange, Y\. Liu, E\. Nie, J\. Strötgen, and H\. Schuetze\(2025\-07\)Lost in multilinguality: dissecting cross\-lingual factual inconsistency in transformer language models\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5075–5094\.External Links:[Link](https://aclanthology.org/2025.acl-long.253/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.253),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p1.1),[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.SSS0.Px1.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[58\]W\. Wang, M\. Wu, B\. Haddow, and A\. Birch\(2025\-07\)Bridging the language gaps in large language models with inference\-time cross\-lingual intervention\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 5418–5433\.External Links:[Link](https://aclanthology.org/2025.acl-long.270/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.270),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[59\]Y\. Wang, Z\. Fan, Q\. Wang, Y\. R\. Fung, and H\. Ji\(2025\-04\)CALM: unleashing the cross\-lingual self\-aligning ability of language model question answering\.InFindings of the Association for Computational Linguistics: NAACL 2025,L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 2809–2817\.External Links:[Link](https://aclanthology.org/2025.findings-naacl.152/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-naacl.152),ISBN 979\-8\-89176\-195\-7Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[60\]Z\. Wang, J\. Huang, J\. Huang, T\. Sun, Y\. Wang, dingqi, H\. Zou, Y\. Liao, X\. Chen, J\. Yao, Y\. Chen, and J\. Su\(2026\)Cross\-lingual data scaling for large language models\.External Links:[Link](https://openreview.net/forum?id=yuRO2wZ8su)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[61\]J\. Wei, X\. Wang, D\. Schuurmans, M\. Bosma, b\. ichter, F\. Xia, E\. Chi, Q\. V\. Le, and D\. Zhou\(2022\)Chain\-of\-thought prompting elicits reasoning in large language models\.InAdvances in Neural Information Processing Systems,S\. Koyejo, S\. Mohamed, A\. Agarwal, D\. Belgrave, K\. Cho, and A\. Oh \(Eds\.\),Vol\.35,pp\. 24824–24837\.External Links:[Link](https://proceedings.neurips.cc/paper_files/paper/2022/file/9d5609613524ecf4f15af0f7b31abca4-Paper-Conference.pdf)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§2\.1](https://arxiv.org/html/2606.26466#S2.SS1.p1.9),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[62\]J\. Wu, J\. Lu, Z\. Ren, G\. Hu, Z\. Wu, D\. Dai, and H\. Wu\(2025\)LLMs are single\-threaded reasoners: demystifying the working mechanism of soft thinking\.External Links:2508\.03440,[Link](https://arxiv.org/abs/2508.03440)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1)\.
- \[63\]L\. Wu, H\. Wei, J\. Tang, S\. Luo, B\. Yang, F\. Huang, Y\. Shen, and W\. Lu\(2026\)MGRPO: unlocking LLM reasoning through multilingual thinking\.External Links:[Link](https://openreview.net/forum?id=QtfALPluAj)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[64\]K\. Xu and I\. Sato\(2026\)A formal comparison between chain of thought and latent thought\.External Links:2509\.25239,[Link](https://arxiv.org/abs/2509.25239)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1)\.
- \[65\]Y\. Xu, X\. Guo, Z\. Zeng, and C\. Miao\(2025\-07\)SoftCoT: soft chain\-of\-thought for efficient reasoning with LLMs\.InProceedings of the 63rd Annual Meeting of the Association for Computational Linguistics \(Volume 1: Long Papers\),W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 23336–23351\.External Links:[Link](https://aclanthology.org/2025.acl-long.1137/),[Document](https://dx.doi.org/10.18653/v1/2025.acl-long.1137),ISBN 979\-8\-89176\-251\-0Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[66\]Y\. Xu, K\. Xu, J\. Zhou, L\. Hu, and L\. Gui\(2025\-11\)Linguistic neuron overlap patterns to facilitate cross\-lingual transfer on low\-resource languages\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 27658–27673\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.1407/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.1407),ISBN 979\-8\-89176\-332\-6Cited by:[§4\.1](https://arxiv.org/html/2606.26466#S4.SS1.SSS0.Px1.p1.1)\.
- \[67\]W\. Xuan, R\. Yang, H\. Qi, Q\. Zeng, Y\. Xiao, A\. Feng, D\. Liu, Y\. Xing, J\. Wang, F\. Gao, J\. Lu, Y\. Jiang, H\. Li, X\. Li, K\. Yu, R\. Dong, S\. Gu, Y\. Li, X\. Xie, F\. Juefei\-Xu, F\. Khomh, O\. Yoshie, Q\. Chen, D\. Teodoro, N\. Liu, R\. Goebel, L\. Ma, E\. Marrese\-Taylor, S\. Lu, Y\. Iwasawa, Y\. Matsuo, and I\. Li\(2025\-11\)MMLU\-ProX: a multilingual benchmark for advanced large language model evaluation\.InProceedings of the 2025 Conference on Empirical Methods in Natural Language Processing,C\. Christodoulopoulos, T\. Chakraborty, C\. Rose, and V\. Peng \(Eds\.\),Suzhou, China,pp\. 1513–1532\.External Links:[Link](https://aclanthology.org/2025.emnlp-main.79/),[Document](https://dx.doi.org/10.18653/v1/2025.emnlp-main.79),ISBN 979\-8\-89176\-332\-6Cited by:[Appendix B](https://arxiv.org/html/2606.26466#A2.p1.1)\.
- \[68\]A\. Yang, A\. Li, B\. Yang, B\. Zhang, B\. Hui, B\. Zheng, B\. Yu, C\. Gao, C\. Huang, C\. Lv, C\. Zheng, D\. Liu, F\. Zhou, F\. Huang, F\. Hu, H\. Ge, H\. Wei, H\. Lin, J\. Tang, J\. Yang, J\. Tu, J\. Zhang, J\. Yang, J\. Yang, J\. Zhou, J\. Zhou, J\. Lin, K\. Dang, K\. Bao, K\. Yang, L\. Yu, L\. Deng, M\. Li, M\. Xue, M\. Li, P\. Zhang, P\. Wang, Q\. Zhu, R\. Men, R\. Gao, S\. Liu, S\. Luo, T\. Li, T\. Tang, W\. Yin, X\. Ren, X\. Wang, X\. Zhang, X\. Ren, Y\. Fan, Y\. Su, Y\. Zhang, Y\. Zhang, Y\. Wan, Y\. Liu, Z\. Wang, Z\. Cui, Z\. Zhang, Z\. Zhou, and Z\. Qiu\(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§D\.1](https://arxiv.org/html/2606.26466#A4.SS1.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p4.1)\.
- \[69\]W\. Yang, J\. Wu, C\. Li, C\. Zong, and J\. Zhang\(2025\)Parallel scaling law: unveiling reasoning generalization through a cross\-linguistic perspective\.External Links:2510\.02272,[Link](https://arxiv.org/abs/2510.02272)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[70\]Z\. Yong, M\. F\. Adilazuarda, J\. Mansurov, R\. Zhang, N\. Muennighoff, C\. Eickhoff, G\. I\. Winata, J\. Kreutzer, S\. H\. Bach, and A\. F\. Aji\(2025\)Crosslingual reasoning through test\-time scaling\.External Links:2505\.05408,[Link](https://arxiv.org/abs/2505.05408)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[71\]H\. Yoo, J\. Jin, K\. Cho, and A\. Oh\(2025\)Code\-switching in\-context learning for cross\-lingual transfer of large language models\.External Links:2510\.05678,[Link](https://arxiv.org/abs/2510.05678)Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1)\.
- \[72\]X\. Zhang, Y\. Liang, F\. Meng, S\. Zhang, K\. Huang, Y\. Chen, J\. Xu, and J\. Zhou\(2026\)Think natively: unlocking multilingual reasoning with consistency\-enhanced reinforcement learning\.External Links:2510\.07300,[Link](https://arxiv.org/abs/2510.07300)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px1.p1.1),[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[73\]Z\. Zhang, X\. He, W\. Yan, A\. Shen, C\. Zhao, and X\. E\. Wang\(2025\)Soft thinking: unlocking the reasoning potential of LLMs in continuous concept space\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=ByQdHPGKgU)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§D\.5](https://arxiv.org/html/2606.26466#A4.SS5.SSS0.Px1.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p2.1),[§2\.1](https://arxiv.org/html/2606.26466#S2.SS1.p3.3),[§2\.2](https://arxiv.org/html/2606.26466#S2.SS2.SSS0.Px1.p1.1),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px2.p1.3),[§3\.1](https://arxiv.org/html/2606.26466#S3.SS1.SSS0.Px4.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.
- \[74\]C\. Zhao, X\. Huang, X\. Han, S\. Huang, C\. Deng, and J\. Feng\(2026\)Align to the pivot: dual alignment with self\-feedback for multilingual math reasoning\.External Links:2601\.17671,[Link](https://arxiv.org/abs/2601.17671)Cited by:[Appendix A](https://arxiv.org/html/2606.26466#A1.SS0.SSS0.Px1.p1.1),[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[75\]R\. Zhao, Y\. Liu, H\. Schütze, and M\. A\. Hedderich\(2025\)A comprehensive evaluation of multilingual chain\-of\-thought reasoning: performance, consistency, and faithfulness across languages\.External Links:2510\.09555,[Link](https://arxiv.org/abs/2510.09555)Cited by:[§C\.1](https://arxiv.org/html/2606.26466#A3.SS1.p1.1),[§1](https://arxiv.org/html/2606.26466#S1.p1.1)\.
- \[76\]Y\. Zhao, W\. Zhang, H\. Wang, K\. Kawaguchi, and L\. Bing\(2025\-04\)AdaMergeX: cross\-lingual transfer with large language models via adaptive adapter merging\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 9785–9800\.External Links:[Link](https://aclanthology.org/2025.naacl-long.493/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.493),ISBN 979\-8\-89176\-189\-6Cited by:[§C\.2](https://arxiv.org/html/2606.26466#A3.SS2.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px1.p1.1)\.
- \[77\]L\. Zheng, L\. Yin, Z\. Xie, C\. L\. Sun, J\. Huang, C\. H\. Yu, S\. Cao, C\. Kozyrakis, I\. Stoica, J\. E\. Gonzalez,et al\.\(2024\)Sglang: efficient execution of structured language model programs\.Advances in neural information processing systems37,pp\. 62557–62583\.Cited by:[§D\.3](https://arxiv.org/html/2606.26466#A4.SS3.p1.4)\.
- \[78\]Y\. Zheng, R\. Zhang, J\. Zhang, Y\. Ye, and Z\. Luo\(2024\-08\)LlamaFactory: unified efficient fine\-tuning of 100\+ language models\.InProceedings of the 62nd Annual Meeting of the Association for Computational Linguistics \(Volume 3: System Demonstrations\),Y\. Cao, Y\. Feng, and D\. Xiong \(Eds\.\),Bangkok, Thailand,pp\. 400–410\.External Links:[Link](https://aclanthology.org/2024.acl-demos.38/),[Document](https://dx.doi.org/10.18653/v1/2024.acl-demos.38)Cited by:[§D\.2](https://arxiv.org/html/2606.26466#A4.SS2.p1.7)\.
- \[79\]H\. Zhu, S\. Hao, Z\. Hu, J\. Jiao, S\. Russell, and Y\. Tian\(2025\)Reasoning by superposition: a theoretical perspective on chain of continuous thought\.InThe Thirty\-ninth Annual Conference on Neural Information Processing Systems,External Links:[Link](https://openreview.net/forum?id=UdOEZgWJLc)Cited by:[§C\.3](https://arxiv.org/html/2606.26466#A3.SS3.p1.1),[§5](https://arxiv.org/html/2606.26466#S5.SS0.SSS0.Px2.p1.1)\.

## Appendix AFurther discussion, limitations, and future work

While our study shows that aligning soft\-token representations during SFT improves multilingual reasoning, it opens several directions for future work\.

#### From SFT\-only alignment to reinforcement learning\.

SOLAR is studied as an auxiliary loss alongside supervised cross\-entropy\. Recent work suggests that reinforcement learning consistently outperforms SFT for cross\-lingual reasoning and generalizes better to unseen languages\[[19](https://arxiv.org/html/2606.26466#bib.bib48)\]\. Future work could integrate soft\-token alignment into preference\- or reward\-based post\-training, for example by combining the alignment loss with Direct Preference Optimization \(DPO\)\[[41](https://arxiv.org/html/2606.26466#bib.bib56)\]or Group Relative Policy Optimization \(GRPO\)\[[45](https://arxiv.org/html/2606.26466#bib.bib38)\]\. This could compound the consistency gains of soft\-token alignment with the generalization that policy optimization brings to unseen languages\. Recent pivot\-based multilingual RL methods such as PASMR\[[74](https://arxiv.org/html/2606.26466#bib.bib15)\], TAPO\[[20](https://arxiv.org/html/2606.26466#bib.bib16)\], and M\-Thinker\[[72](https://arxiv.org/html/2606.26466#bib.bib33)\]reward consistency between English and target\-language reasoning at the output level\. SOLAR’s soft\-token alignment offers a complementary representation\-level signal that could be added to these reward\-based objectives as an auxiliary alignment term during policy optimization\.

#### Reasoning domains beyond math and science\.

Our evaluation focuses on multilingual mathematical reasoning \(MGSM, AIME 2024, AIME 2025\) and graduate\-level scientific question answering \(GPQA Diamond\), where final answers are short and verifiable\. Whether response\-level mean\-pooled soft\-token summaries carry the same cross\-lingual signal for tasks with open\-ended outputs \(e\.g\.,multilingual summarization\[[13](https://arxiv.org/html/2606.26466#bib.bib37)\]\) remains an open question\. Future work could extend SOLAR to broader generation tasks and investigate step\-level or chunk\-level alignment for reasoning traces whose semantic structure is non\-monotonic or interleaved with retrieval\.

#### Scaling parallel reasoning data across languages\.

SOLAR is trained on M\-s1k\[[1](https://arxiv.org/html/2606.26466#bib.bib76)\], which provides parallel long CoT traces in 10 languages\. We report results on the 7 that overlap with our evaluation suites \(Appendix[D\.4](https://arxiv.org/html/2606.26466#A4.SS4)\)\. Expanding coverage requires translating long CoT traces into each new language with mathematical\-fidelity verification, which incurs non\-trivial cost\. SOLAR’s effectiveness on a broader set of languages, particularly low\-resource ones, therefore remains to be validated\. One direction is to scale parallel reasoning data through verified translation pipelines\. Another is to investigate whether soft\-token alignment can be combined with weaker supervision, such as machine\-translated traces with quality filtering, or unpaired monolingual reasoning anchored through a shared retrieval signal, to relax the strict parallel\-data requirement\.

#### Toward broader model families and scales\.

Our main fine\-tuning experiments target Qwen3\-4B/8B and Gemma3\-4B/12B\. Although the representation analysis \(§[4\.1](https://arxiv.org/html/2606.26466#S4.SS1)\) covers 10 models from the Qwen3, Gemma3, and DeepSeek\-R1\-Distill families, the SFT pipeline is validated on two families at moderate scales\. Larger models \(e\.g\.,30B\+\), mixture\-of\-experts architectures, and multimodal LLMs may exhibit different vocabulary geometries and pre\-training compositions, potentially altering the strength and shape of the soft\-token alignment signal\. Extending SOLAR to a broader set of families and scales offers a route to assessing how generally the soft\-token alignment principle holds\.

## Appendix BBroader impacts

As multilingual LLMs serve users worldwide, ensuring consistent reasoning across languages is both a technical and an equity concern\. Most reasoning gains in modern LLMs remain concentrated in English and a handful of high\-resource languages\[[67](https://arxiv.org/html/2606.26466#bib.bib36)\], leaving speakers of underrepresented languages with weaker access to advanced reasoning in domains such as decision support\[[34](https://arxiv.org/html/2606.26466#bib.bib34)\]\. Our work targets this gap by aligning soft\-token representations across languages during SFT, with the largest gains on low\-resource languages \(e\.g\.,Swahili MGSM accuracy rises from 29\.04% to 53\.48% for Qwen3\-4B, and Telugu AIME 2024 from 5\.67% to 29\.67%\), narrowing performance disparities while preserving native\-language generation \(Appendix[F\.3](https://arxiv.org/html/2606.26466#A6.SS3)\)\.

Beyond accuracy, our analysis helps make multilingual model behavior more interpretable and controllable\. Soft\-token embeddings provide a continuous, vocabulary\-shared signal in which cross\-lingual reasoning can be inspected and shaped, complementing hidden\-state probing and offering a lightweight, architecture\-agnostic mechanism that can be combined with other multilingual training pipelines\. Because SOLAR adds no architectural modification and reuses the model’s own forward\-pass logits, it is also accessible when compute or model\-modification budgets are limited\.

At the same time, anchoring all non\-English summaries to an English pivot raises questions about cultural and linguistic representation\. An English\-anchored shared space may carry English\-centric reasoning conventions, framing assumptions, or normative answers that surface only on culturally grounded tasks not covered by our benchmarks\[[12](https://arxiv.org/html/2606.26466#bib.bib35)\]\. Stronger multilingual reasoning also extends the reach of downstream misuse currently bottlenecked by English\-only proficiency, and our work does not address multilingual safety alignment\. We see multi\-pivot or pivot\-free alignment, evaluation on culturally grounded reasoning, and multilingual safety post\-training as important complements to our work\. Overall, we hope our work contributes to a more inclusive and principled approach to multilingual reasoning and helps narrow the cross\-lingual capability gap in a way that is mindful of the linguistic communities these systems aim to serve\.

## Appendix CRelated work

### C\.1Multilingual reasoning

LLMs trained on multilingual corpora exhibit cross\-lingual reasoning abilities\[[53](https://arxiv.org/html/2606.26466#bib.bib42),[68](https://arxiv.org/html/2606.26466#bib.bib1),[75](https://arxiv.org/html/2606.26466#bib.bib2)\], yet show inconsistent performance across different languages\[[39](https://arxiv.org/html/2606.26466#bib.bib6),[57](https://arxiv.org/html/2606.26466#bib.bib4)\]\. Specifically, given the same task, LLMs generally perform better when prompted in English, whereas their reasoning performance deteriorates when the task is expressed in low\-resource languages\. Recent work examines these disparities through internal model mechanisms\.Wanget al\.\[[57](https://arxiv.org/html/2606.26466#bib.bib4)\]reveal that models encode factual knowledge in a language\-agnostic concept space through intermediate layers, but errors arise during the final\-layer transition to language\-specific representations\.Liuet al\.\[[31](https://arxiv.org/html/2606.26466#bib.bib3)\]identify language\-specific attention heads that enable cross\-lingual attention transfer\. Recently,Limet al\.\[[27](https://arxiv.org/html/2606.26466#bib.bib81)\]show that larger models increasingly rely on language\-specific representations, hindering cross\-lingual transfer\. These mechanistic insights provide a foundation for understanding and improving cross\-lingual reasoning\.

### C\.2Cross\-lingual alignment

Efforts to improve multilingual reasoning in LLMs can be broadly categorized into training\-based and inference\-based approaches\[[14](https://arxiv.org/html/2606.26466#bib.bib43)\]\. Training\-based methods use translated parallel data during pre\-training\[[40](https://arxiv.org/html/2606.26466#bib.bib44),[11](https://arxiv.org/html/2606.26466#bib.bib77),[60](https://arxiv.org/html/2606.26466#bib.bib70)\]or post\-training\[[23](https://arxiv.org/html/2606.26466#bib.bib80),[24](https://arxiv.org/html/2606.26466#bib.bib79),[7](https://arxiv.org/html/2606.26466#bib.bib63),[28](https://arxiv.org/html/2606.26466#bib.bib51),[21](https://arxiv.org/html/2606.26466#bib.bib78),[1](https://arxiv.org/html/2606.26466#bib.bib76),[69](https://arxiv.org/html/2606.26466#bib.bib58)\], language\-mixed CoT data\[[5](https://arxiv.org/html/2606.26466#bib.bib61),[50](https://arxiv.org/html/2606.26466#bib.bib68)\], adapter merging\[[76](https://arxiv.org/html/2606.26466#bib.bib64)\], representation alignment\[[35](https://arxiv.org/html/2606.26466#bib.bib62),[29](https://arxiv.org/html/2606.26466#bib.bib60),[2](https://arxiv.org/html/2606.26466#bib.bib7),[3](https://arxiv.org/html/2606.26466#bib.bib57)\], and preference optimization methods without parallel data\[[46](https://arxiv.org/html/2606.26466#bib.bib55),[59](https://arxiv.org/html/2606.26466#bib.bib50),[25](https://arxiv.org/html/2606.26466#bib.bib52),[10](https://arxiv.org/html/2606.26466#bib.bib46),[26](https://arxiv.org/html/2606.26466#bib.bib75),[30](https://arxiv.org/html/2606.26466#bib.bib47),[63](https://arxiv.org/html/2606.26466#bib.bib45)\]\.Huanget al\.\[[19](https://arxiv.org/html/2606.26466#bib.bib48)\]demonstrate that RL consistently outperforms SFT in cross\-lingual reasoning and achieves better generalization even when trained on non\-English data\. PASMR\[[74](https://arxiv.org/html/2606.26466#bib.bib15)\]maps multilingual inputs to an English pivot via SFT and then runs self\-feedback RL that rewards consistency between pivot and target\-language reasoning\. TAPO\[[20](https://arxiv.org/html/2606.26466#bib.bib16)\]couples a translation\-quality reward with the standard reasoning reward under a step\-level relative\-advantage mechanism that decouples credit assignment between translation and reasoning tokens\. Recently, M\-Thinker\[[72](https://arxiv.org/html/2606.26466#bib.bib33)\]combines a language\-consistency reward with a cross\-lingual thinking\-alignment reward computed via LLM\-as\-judge against an English reasoning trace\. These methods are complementary to SOLAR’s soft\-token alignment signal, and we discuss their potential integration in Appendix[A](https://arxiv.org/html/2606.26466#A1)\. Inference\-based methods include in\-context learning\[[52](https://arxiv.org/html/2606.26466#bib.bib66),[71](https://arxiv.org/html/2606.26466#bib.bib65)\], cross\-lingual activation steering\[[17](https://arxiv.org/html/2606.26466#bib.bib67),[37](https://arxiv.org/html/2606.26466#bib.bib74)\], test\-time scaling\[[49](https://arxiv.org/html/2606.26466#bib.bib59),[70](https://arxiv.org/html/2606.26466#bib.bib69)\], and learned linear transformations between source\- and target\-language hidden states applied at inference\[[58](https://arxiv.org/html/2606.26466#bib.bib14)\]\. Studies show that prompting models to reason in English generally preserves or improves non\-English task performance\[[1](https://arxiv.org/html/2606.26466#bib.bib76),[70](https://arxiv.org/html/2606.26466#bib.bib69)\]\. SOLAR differs from these inference\-time interventions in that it modifies the model parameters during SFT through an auxiliary soft\-token alignment loss, with no per\-language transformation at inference and no test\-time overhead beyond standard discrete decoding\.

### C\.3Continuous\-space reasoning

While CoT prompting\[[61](https://arxiv.org/html/2606.26466#bib.bib41)\]enhances LLM reasoning through step\-by\-step discrete linguistic tokens, recent work argues that LLMs should mirror human cognition, which operates through abstract, continuous concepts rather than explicit verbalized steps\[[6](https://arxiv.org/html/2606.26466#bib.bib8)\]\. Several methods train LLMs to reason in continuous space\[[65](https://arxiv.org/html/2606.26466#bib.bib29),[36](https://arxiv.org/html/2606.26466#bib.bib40),[47](https://arxiv.org/html/2606.26466#bib.bib9)\]\. COCONUT\[[18](https://arxiv.org/html/2606.26466#bib.bib39)\]feeds the last hidden state as the next input embedding, enabling the model to encode multiple reasoning steps simultaneously\. For training\-free approaches, Soft Thinking\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\]replaces discrete token embeddings with probability\-weighted soft tokens, forming a continuous concept space for richer representations at inference time\.Wuet al\.\[[62](https://arxiv.org/html/2606.26466#bib.bib23)\]enhance this with noise via Dirichlet resampling and the Gumbel\-Softmax trick\.Buttet al\.\[[4](https://arxiv.org/html/2606.26466#bib.bib17)\]train continuous CoTs with reinforcement learning by injecting Gaussian noise into the input embedding mixture to enable exploration, removing the need to distill from ground\-truth discrete reasoning traces\. They report that continuous\-CoT training matches discrete training at pass@1 and surpasses it at pass@32 on math benchmarks, and that decoding with discrete tokens at inference on top of a continuous\-CoT trained model performs best\. Theoretical frameworks further demonstrate that continuous CoT can solve graph reachability problems more efficiently by maintaining multiple search frontiers simultaneously\[[16](https://arxiv.org/html/2606.26466#bib.bib22),[79](https://arxiv.org/html/2606.26466#bib.bib21)\]\. Together, these works show that continuous\-space reasoning enables richer information encoding compared to discrete token\-based methods\[[51](https://arxiv.org/html/2606.26466#bib.bib28),[64](https://arxiv.org/html/2606.26466#bib.bib24)\]\. While prior work focuses on monolingual reasoning, SOLAR repurposes the soft\-token formulation\[[4](https://arxiv.org/html/2606.26466#bib.bib17),[73](https://arxiv.org/html/2606.26466#bib.bib5)\]as a cross\-lingual alignment signal during post\-training\. We investigate whether probability\-weighted mixtures over the shared vocabulary embedding matrix can serve as a comparable signal across languages, even when their surface tokens are disjoint\.

## Appendix DImplementation details

### D\.1Models

For main experiments \(§[3](https://arxiv.org/html/2606.26466#S3)\), we fine\-tune Qwen3\-4B/8B\[[68](https://arxiv.org/html/2606.26466#bib.bib1)\]and Gemma3\-4B/12B\[[53](https://arxiv.org/html/2606.26466#bib.bib42)\]via full\-parameter SFT on M\-s1k\. We center the headline analysis on the Qwen3 family for its strong multilingual reasoning and wide adoption; full Gemma3 results \(MGSM and GPQA\) appear in §[E](https://arxiv.org/html/2606.26466#A5)\. We use the 4B and 8B scales for Qwen3 because they balance reasoning capability against the compute cost of full\-parameter fine\-tuning on A40 GPUs \(48 GB\)\. For representation experiments \(§[4\.1](https://arxiv.org/html/2606.26466#S4.SS1)\), we select 10 models from three families: Qwen3, Gemma3, and DeepSeek\-R1\-Distill\[[9](https://arxiv.org/html/2606.26466#bib.bib32)\]\. These span multiple scales \(1B–27B\) and both base and instruction\-tuned variants \(e\.g\.,Qwen3\-4B\-Basevs\.Qwen3\-4B\)\. Specifically, Qwen3 and Gemma3 are advanced multilingual models supporting 119 and 140\+ languages, respectively\.

### D\.2Training setup

All main\-experiment models undergo full\-parameter fine\-tuning using theLLaMA\-Factoryframework\[[78](https://arxiv.org/html/2606.26466#bib.bib71)\]\. Qwen3\-4B is trained in non\-thinking mode, while Qwen3\-8B is trained in thinking mode\. We train for 5 epochs with a batch size of 32\. All runs use bfloat16 precision with a learning rate of1×10−51\\times 10^\{\-5\}that warms up linearly over the first 10% of training and then decays to 0 following a cosine schedule\. We use the AdamW optimizer\[[32](https://arxiv.org/html/2606.26466#bib.bib27)\]withβ1=0\.9\\beta\_\{1\}=0\.9andβ2=0\.999\\beta\_\{2\}=0\.999\. The maximum sequence length is 16,384 tokens\. We use DeepSpeed ZeRO Stage 2\[[42](https://arxiv.org/html/2606.26466#bib.bib26)\]to reduce memory usage\. All experiments are conducted on NVIDIA A40 GPUs \(48 GB\)\. For the SOLAR alignment objective, we compute soft tokens withτ=1\.0\\tau\{=\}1\.0and top\-k=30k\{=\}30\. We ablate the alignment weightλ∈\{0\.25,0\.50,0\.75,1\.00\}\\lambda\\in\\\{0\.25,0\.50,0\.75,1\.00\\\}and soft\-token temperatureτ∈\{0\.6,1\.0,2\.0\}\\tau\\in\\\{0\.6,1\.0,2\.0\\\}in §[4\.3](https://arxiv.org/html/2606.26466#S4.SS3)\. We employ a group\-aware batch sampler that places all translations of each problem contiguously, enabling co\-occurrence within the same training step even under small per\-device batch sizes\.

### D\.3Evaluation setup

We useSGLang\(v0\.4\.6\)\[[77](https://arxiv.org/html/2606.26466#bib.bib30)\]as the inference backend\. For all evaluations, we use a sampling temperature of0\.60\.6, top\-k=30k\{=\}30, top\-p=0\.95p\{=\}0\.95, and a maximum generation length of 32,768 tokens\. Note that this sampling temperature is used solely for next\-token decoding at inference time and is distinct from the soft\-token temperatureτ\\tauused during SOLAR training \(Eq\.[2](https://arxiv.org/html/2606.26466#S2.E2)–[3](https://arxiv.org/html/2606.26466#S2.E3)\)\. We run each evaluation multiple times and report the average to reduce estimation variance: five runs for MGSM and GPQA, and ten runs for AIME 2024 and AIME 2025\. We count a model’s answer as correct only if the extracted final answer \(i\.e\.,the content within\\boxed\{\}\) matches the ground\-truth label\.

### D\.4Datasets

#### Training data\.

M\-s1k\[[1](https://arxiv.org/html/2606.26466#bib.bib76)\]is a multilingual long CoT reasoning dataset translated from s1k\[[33](https://arxiv.org/html/2606.26466#bib.bib49)\]by Gemini\-2\.0\-Flash\[[15](https://arxiv.org/html/2606.26466#bib.bib25)\]\. It contains 1,000 parallel problems across 10 languages \(English \(En\), Chinese \(Zh\), French \(Fr\), Japanese \(Ja\), Latvian \(Lv\), Swahili \(Sw\), Telugu \(Te\), Thai \(Th\), Afrikaans \(Af\), and Marathi \(Mr\)\), totaling 10,000 training examples\. Each pairs a question with a detailed reasoning trace and final solution\. We additionally annotate each with aparallel\_idlinking translations of the same problem across languages and alanguagetag for fine\-tuning\.

#### Evaluation benchmarks\.

We evaluate on four benchmarks covering multilingual mathematical reasoning and science question answering\. MGSM \(Multilingual Grade School Math\)\[[48](https://arxiv.org/html/2606.26466#bib.bib10)\]extends GSM8K\[[8](https://arxiv.org/html/2606.26466#bib.bib11)\]with 250 parallel grade\-school math word problems requiring multi\-step arithmetic\. AIME 2024\[[56](https://arxiv.org/html/2606.26466#bib.bib18)\]and AIME 2025\[[22](https://arxiv.org/html/2606.26466#bib.bib19)\]each contain 30 competition\-level problems from the American Invitational Mathematics Examination\. GPQA Diamond\[[43](https://arxiv.org/html/2606.26466#bib.bib13)\]is a graduate\-level, Google\-proof benchmark of 198 multiple\-choice questions across physics, chemistry, and biology, designed to challenge domain experts\. The multilingual versions of AIME 2024, AIME 2025, and GPQA Diamond are from XReasoning\[[38](https://arxiv.org/html/2606.26466#bib.bib20)\]\. All four cover 11 languages: English \(En\), Spanish \(Es\), French \(Fr\), German \(De\), Russian \(Ru\), Chinese \(Zh\), Japanese \(Ja\), Thai \(Th\), Swahili \(Sw\), Bengali \(Bn\), and Telugu \(Te\)\. Since only 7 of the 10 M\-s1k languages overlap with the evaluation set \(En, Fr, Ja, Sw, Te, Th, Zh\), we report main results on these seven\. For the representation analysis \(§[4\.1](https://arxiv.org/html/2606.26466#S4.SS1)\), we sample 50 instances per language from MGSM and GPQA Diamond and use all 30 per subset for AIME 2024 and AIME 2025; the main experiments \(§[3](https://arxiv.org/html/2606.26466#S3)\) use all instances\. Expanding to all 11 languages would require translating s1k reasoning traces, which incurs non\-trivial API cost for long CoT data and needs quality verification to ensure mathematical fidelity across languages\. We leave broader language expansion to future work\.

### D\.5Baselines

#### Soft Thinking\.

We follow the formulation in §[2\.1](https://arxiv.org/html/2606.26466#S2.SS1)and apply inference\-only soft thinking\[[73](https://arxiv.org/html/2606.26466#bib.bib5)\]to base models\. We use the same decoding hyperparameters as our evaluation setup \(§[D\.3](https://arxiv.org/html/2606.26466#A4.SS3)\)\.

#### MidAlign\.

A middle\-layer contrastive alignment objective that encourages cross\-lingual similarity by maximizing cosine similarity between mean\-pooled hidden states of parallel translation pairs while minimizing similarity to non\-translations within each batch\[[29](https://arxiv.org/html/2606.26466#bib.bib60)\]\. We adapt MidAlign from its original LoRA\-based setup to our full\-parameter SFT setting and apply the contrastive alignment loss at the middle layer \(18 for Qwen3\-4B\) jointly with the cross\-entropy loss, using the same M\-s1k parallel training data and hyperparameters as SOLAR \(§[D\.2](https://arxiv.org/html/2606.26466#A4.SS2)\)\. MidAlign aligns discrete hidden states at intermediate layers, whereas SOLAR aligns soft\-token embeddings\.

#### MAPO\.

An optimization framework that uses a pre\-trained machine translation model to score the consistency between non\-English and English reasoning chains, treating these scores as signals for preference training\[[46](https://arxiv.org/html/2606.26466#bib.bib55)\]\. MAPO optimizes cross\-lingual consistency at the output level via preference learning over discrete reasoning, whereas our method aligns soft\-token embeddings at the representation level during SFT\. For each of the 9,000 non\-English questions in M\-s1k, we samplen=20n\{=\}20candidate outputs from Qwen3\-4B with sampling temperature1\.01\.0\. We then score each output using NLLB\-200\-distilled\-600M\[[54](https://arxiv.org/html/2606.26466#bib.bib54)\]\. The alignment score for each non\-English output is computed as1/ℒCE1/\\mathcal\{L\}\_\{\\text\{CE\}\}, whereℒCE\\mathcal\{L\}\_\{\\text\{CE\}\}is the cross\-entropy loss of the NLLB model translating the non\-English output to the corresponding English reference answer\. Preference pairs are constructed by selecting, for each question, a chosen response that is both correct and has a higher alignment score, paired with a rejected response that has a lower alignment score\. This yields 6,305 preference pairs from 1,148 questions; the remaining questions are excluded because none of the 20 sampled outputs produce a correct answer\. We perform full\-parameter DPO\[[41](https://arxiv.org/html/2606.26466#bib.bib56)\]withβ=0\.1\\beta\{=\}0\.1\. All other training hyperparameters follow the same setup as SOLAR \(§[D\.2](https://arxiv.org/html/2606.26466#A4.SS2)\)\.

#### AlignX\.

A two\-stage framework for multilingual representation alignment\[[2](https://arxiv.org/html/2606.26466#bib.bib7)\]\. Stage 1 combines three objectives: \(1\) an InfoNCE\-style contrastive loss that aligns mean\-pooled hidden states at an intermediate layer across parallel translation pairs using cosine similarity \(with temperature0\.10\.1\), \(2\) a language\-matching classifier at the final layer to reduce off\-target generation, and \(3\) the standard next\-token prediction loss\. Stage 2 performs multilingual instruction fine\-tuning with only the NTP loss\. We apply contrastive alignment at the middle layer of the model\. The language\-matching classifier is a 2\-layer MLP with intermediate dimension 128\. We useα1=0\.3\\alpha\_\{1\}\{=\}0\.3andα2=0\.4\\alpha\_\{2\}\{=\}0\.4as the loss weights for the contrastive and language matching losses, respectively, following the original paper\. All other training hyperparameters follow SOLAR \(§[D\.2](https://arxiv.org/html/2606.26466#A4.SS2)\)\.

## Appendix EExperimental results

Tables[11](https://arxiv.org/html/2606.26466#A5.T11)–[14](https://arxiv.org/html/2606.26466#A5.T14)report per\-language accuracy and cross\-lingual consistency for Qwen3\-4B, Qwen3\-8B, Gemma3\-4B, and Gemma3\-12B, along with their fine\-tuned variants, across four benchmarks\. On MGSM and AIME, SOLAR’s accuracy gains over the base model are largest for low\-resource languages, while high\-resource languages change far less \(within roughly±\\pm1 point\) and occasionally regress\. For example, Qwen3\-4B Swahili improves by \+24\.4 points on MGSM \(\+22\.0 for Qwen3\-8B\), and Qwen3\-4B Telugu by \+24\.0 on AIME 2024\. On GPQA, all seven Qwen3\-4B languages improve under SOLAR, including English, showing broad SFT\-level benefits on this science benchmark\. For Gemma3, MGSM and GPQA Diamond follow the Qwen3 patterns: SOLAR yields the highest average accuracy and CLC at both scales, with low\-resource languages benefiting most\. Baseline strength differs across families: Gemma3\-12B exceeds Qwen3\-8B on MGSM \(89\.50% vs\. 83\.42%\) while Gemma3\-4B trails Qwen3\-4B \(53\.31% vs\. 74\.01%\)\. On GPQA, base Gemma3 performs near or below the four\-way random baseline of 25% \(16\.34% at 4B, 26\.61% at 12B\), and SOLAR raises both scales over SFT\. Because Gemma3\-4B’s GPQA accuracy already sits near the random floor, we omit Gemma3 from AIME 2024 and 2025\. Among Qwen3\-4B baselines, MidAlign is strongest on AIME 2024, while MAPO posts high single\-language MGSM scores without consistent CLC gains, so preference optimization can raise individual\-language accuracy without aligning cross\-lingual reasoning\. MAPO’s English accuracy falls below the base model on MGSM and both AIME sets, as expected when training only on non\-English preference pairs, and its limited signal \(6,305 pairs from 1,148 questions\) likely constrains low\-resource gains\.

Table[15](https://arxiv.org/html/2606.26466#A5.T15)reports the Pearson correlation between average accuracy and CLC across model variants\. The two are nearly perfectly correlated \(r≥0\.997r\\geq 0\.997on every individual benchmark andr=0\.995r=0\.995pooled across all\), so accuracy gains reflect stronger cross\-lingual consistency rather than improvement in a single dominant language\.

Table 11:Per\-language accuracy, average accuracy, and cross\-lingual consistency \(%\) on MGSM for Qwen3 and Gemma3 models and their fine\-tuned variants\.ModelEnFrJaSwTeThZhAvg\(↑\\uparrow\)CLC\(↑\\uparrow\)Qwen3\-4B93\.3681\.3680\.5629\.0464\.2483\.3686\.1674\.0169\.31\+ Soft Thinking93\.4083\.5281\.8428\.2461\.6084\.1286\.9274\.2370\.37\+ SFT94\.0085\.6077\.2047\.2063\.5279\.3686\.8876\.2572\.29\+ MidAlign93\.2485\.1679\.9248\.7263\.0477\.6087\.4476\.4573\.11\+ MAPO92\.2485\.7680\.0051\.2865\.9279\.0488\.4077\.5273\.90\+ AlignX93\.1284\.5680\.3253\.4864\.3280\.4087\.6077\.6973\.00\+ SOLAR93\.3286\.3281\.0053\.4865\.3680\.5687\.8478\.2775\.61Qwen3\-8B94\.8886\.8084\.9659\.3680\.0889\.3688\.4883\.4280\.51\+ Soft Thinking94\.9686\.8084\.8858\.3281\.4488\.0086\.8883\.0480\.01\+ SFT95\.4489\.6082\.2479\.7680\.5691\.2089\.5286\.9084\.71\+ SOLAR94\.9691\.2882\.7281\.3683\.2092\.8091\.6088\.2787\.28Gemma3\-4B68\.6459\.0449\.4433\.9248\.6454\.0859\.4453\.3145\.29\+ Soft Thinking68\.2458\.5650\.3235\.8449\.3653\.0457\.8453\.3145\.01\+ SFT71\.3666\.6454\.0845\.9248\.9657\.8457\.7657\.5153\.12\+ MidAlign71\.3667\.5254\.2447\.5251\.2858\.9658\.7258\.5153\.28\+ MAPO72\.1667\.2054\.3246\.8851\.6858\.1662\.0058\.9154\.32\+ AlignX72\.1667\.9254\.5647\.2051\.6059\.5259\.4458\.9153\.84\+ SOLAR71\.6068\.5654\.4048\.4053\.3659\.2863\.9259\.9355\.04Gemma3\-12B97\.1291\.3687\.8483\.6086\.6489\.8490\.0889\.5086\.95\+ Soft Thinking96\.4891\.6087\.4484\.0088\.4890\.4089\.2889\.6786\.75\+ SFT96\.8094\.1687\.2086\.8890\.8089\.9292\.3291\.1587\.76\+ SOLAR97\.0494\.1687\.4487\.5289\.7691\.4493\.2091\.5189\.52Table 12:Per\-language accuracy, average accuracy, and cross\-lingual consistency \(%\) on AIME 2024 for Qwen3 models and their fine\-tuned variants\.ModelEnFrJaSwTeThZhAvg\(↑\\uparrow\)CLC\(↑\\uparrow\)Qwen3\-4B53\.3343\.6736\.6711\.005\.6726\.0041\.3331\.1023\.89\+ Soft Thinking53\.3351\.3330\.0010\.006\.3326\.0046\.0031\.8624\.44\+ SFT45\.3346\.3334\.3322\.0020\.6736\.0038\.6734\.7625\.17\+ MidAlign44\.3349\.6735\.6727\.6728\.6736\.3334\.3336\.6727\.11\+ MAPO46\.3345\.3330\.6722\.3323\.0036\.0039\.6734\.7625\.89\+ AlignX45\.6746\.3335\.6726\.3324\.0032\.3341\.6736\.0026\.88\+ SOLAR45\.3350\.6734\.3332\.6729\.6738\.6738\.3338\.5229\.67Qwen3\-8B75\.3373\.6770\.3337\.3352\.6768\.6756\.6762\.1056\.06\+ Soft Thinking76\.0073\.0070\.3337\.6754\.0069\.6757\.0062\.5256\.91\+ SFT72\.0077\.3368\.3355\.3361\.6772\.0053\.0065\.6758\.67\+ SOLAR72\.6778\.3368\.6758\.3363\.0075\.0056\.6767\.5260\.61Table 13:Per\-language accuracy, average accuracy, and cross\-lingual consistency \(%\) on AIME 2025 for Qwen3 models and their fine\-tuned variants\.ModelEnFrJaSwTeThZhAvg\(↑\\uparrow\)CLC\(↑\\uparrow\)Qwen3\-4B36\.6731\.6727\.009\.333\.3322\.3331\.0023\.0517\.61\+ Soft Thinking30\.0036\.6730\.336\.673\.3325\.0031\.3323\.3318\.22\+ SFT35\.6733\.3326\.0021\.0017\.0030\.0032\.3327\.9020\.44\+ MidAlign31\.6734\.3327\.6723\.6720\.0032\.0034\.0029\.0523\.22\+ MAPO32\.6736\.6724\.3313\.339\.6728\.6735\.3325\.8118\.94\+ AlignX36\.6734\.6725\.0022\.6717\.3326\.0032\.0027\.7620\.10\+ SOLAR35\.3337\.3326\.6726\.3323\.3331\.6736\.0030\.9524\.51Qwen3\-8B63\.0066\.6760\.0041\.6743\.6755\.0054\.3354\.9145\.06\+ Soft Thinking61\.0065\.6761\.0040\.3349\.0053\.6752\.3354\.7144\.22\+ SFT60\.6767\.3360\.6748\.6752\.3363\.0056\.0058\.3850\.13\+ SOLAR61\.0069\.0059\.3351\.0055\.6765\.6759\.0060\.1053\.02Table 14:Per\-language accuracy, average accuracy, and cross\-lingual consistency \(%\) on GPQA Diamond for Qwen3 and Gemma3 models and their fine\-tuned variants\.ModelEnFrJaSwTeThZhAvg\(↑\\uparrow\)CLC\(↑\\uparrow\)Qwen3\-4B45\.4529\.8016\.5719\.8014\.2430\.3036\.4627\.5214\.56\+ Soft Thinking48\.4829\.9014\.5519\.3913\.0329\.1936\.8727\.3414\.31\+ SFT58\.0850\.6135\.8637\.8830\.1044\.2449\.4943\.7533\.15\+ MidAlign54\.7549\.6034\.5536\.4634\.5540\.6150\.5143\.0032\.47\+ MAPO47\.1747\.5834\.1434\.6530\.6141\.4146\.6740\.3230\.15\+ AlignX56\.0649\.4934\.7534\.5533\.9440\.6151\.8243\.0332\.97\+ SOLAR58\.0853\.8434\.2439\.0933\.2344\.9553\.2345\.2436\.26Qwen3\-8B58\.3857\.3752\.9345\.6648\.6954\.1451\.6252\.6842\.69\+ Soft Thinking61\.3157\.8853\.0344\.2447\.6854\.7552\.5353\.0643\.42\+ SFT69\.4964\.1458\.5962\.6361\.3162\.7366\.5763\.6455\.73\+ SOLAR68\.9966\.1661\.1162\.0263\.2364\.6568\.5964\.9657\.27Gemma3\-4B17\.4717\.8814\.6513\.8414\.1418\.4817\.8816\.345\.37\+ Soft Thinking16\.2616\.9716\.6714\.8514\.9517\.5816\.5716\.265\.35\+ SFT33\.7431\.2128\.5927\.9830\.7128\.5927\.2729\.7315\.45\+ MidAlign33\.5431\.3128\.3827\.6830\.5129\.0928\.3829\.8415\.56\+ MAPO32\.2230\.8128\.8927\.0728\.9929\.6027\.6829\.3214\.85\+ AlignX32\.8331\.4127\.3725\.5629\.0928\.2826\.8728\.7714\.14\+ SOLAR33\.4333\.3330\.1028\.8929\.8029\.6028\.3830\.5016\.26Gemma3\-12B30\.5130\.5123\.2328\.5923\.0327\.5822\.8326\.6113\.43\+ Soft Thinking29\.6028\.0821\.2128\.7925\.1528\.4822\.3226\.2312\.27\+ SFT43\.1344\.5529\.8037\.6826\.1639\.6036\.7736\.8130\.61\+ SOLAR47\.0746\.9731\.6239\.7030\.5143\.7436\.7739\.4834\.85Table 15:Pearson correlation between average accuracy and CLC across model variants\.BenchmarkPearsonrrpp\-valueMGSM0\.9973\.5×10−103\.5\\times 10^\{\-10\}AIME 20240\.9985\.1×10−115\.1\\times 10^\{\-11\}AIME 20250\.9974\.9×10−104\.9\\times 10^\{\-10\}GPQA0\.9992\.1×10−122\.1\\times 10^\{\-12\}All0\.9951\.4×10−391\.4\\times 10^\{\-39\}
## Appendix FAnalysis

### F\.1Ablation study

#### Alignment weightλ\\lambda\.

Table[16](https://arxiv.org/html/2606.26466#A6.T16)reportsλ∈\{0\.25,0\.50,0\.75,1\.00\}\\lambda\\in\\\{0\.25,0\.50,0\.75,1\.00\\\}withτ=1\.0\\tau\{=\}1\.0across all four benchmarks\. The weight trades off cross\-entropy against the alignment loss \(Eq\.[5](https://arxiv.org/html/2606.26466#S2.E5)\)\.λ=0\.25\\lambda\{=\}0\.25attains the highest MGSM accuracy and CLC, withλ=0\.50\\lambda\{=\}0\.50close behind, but we adoptλ=0\.50\\lambda\{=\}0\.50because it leads on the harder benchmarks \(e\.g\.,38\.52% vs\. 37\.43% on AIME 2024\)\. Performance degrades atλ=0\.75\\lambda\{=\}0\.75and falls below the base model atλ=1\.00\\lambda\{=\}1\.00\(69\.25% vs\. 74\.01% on MGSM\), where low\-resource languages lose the most \(Swahili AIME 2024: 32\.67%→\\rightarrow10\.00%\), indicating that excessive alignment overwhelms cross\-entropy and disrupts languages with sparser pre\-training coverage\.

#### Soft\-token temperatureτ\\tau\.

Table[17](https://arxiv.org/html/2606.26466#A6.T17)\(a\) reportsτ∈\{0\.6,1\.0,2\.0\}\\tau\\in\\\{0\.6,1\.0,2\.0\\\}withλ=0\.50\\lambda\{=\}0\.50\. Lowerτ\\tauproduces more peaked distributions that approximate discrete token embeddings, while higherτ\\tauyields smoother mixtures over more vocabulary tokens\. Atτ=0\.6\\tau\{=\}0\.6, average accuracy and CLC drop sharply \(by 17\.8 and 18\.6 points relative toτ=1\.0\\tau\{=\}1\.0\), and the loss is highly uneven: Telugu falls to 14\.48% while English remains high at 89\.90%\. With the continuous signal gone, the objective reduces to aligning discrete representations, so low\-resource languages, which have the weakest baseline alignment, suffer the most\. Atτ=2\.0\\tau\{=\}2\.0the drop is moderate \(−\-1\.9 accuracy,−\-2\.5 CLC\) and the per\-language pattern partially reverses: Swahili and Telugu reach their best accuracies across the three temperatures while high\-resource languages decline \(e\.g\.,French 86\.32%→\\rightarrow79\.28%\)\. Smoother distributions thus spread the signal more broadly, helping sparse\-coverage languages at the cost of the semantic specificity high\-resource languages rely on\. The optimum atτ=1\.0\\tau\{=\}1\.0balances the two: soft tokens must differ meaningfully from discrete tokens yet stay specific enough for effective alignment\.

#### Alignment representation space\.

Table[17](https://arxiv.org/html/2606.26466#A6.T17)\(b\) compares aligning mean\-pooled soft\-token embeddings inℝd\\mathbb\{R\}^\{d\}\(SOLAR\) against mean\-pooled probability distributions inℝ\|V\|\\mathbb\{R\}^\{\|V\|\}, which bypasses the embedding layer and minimizes cosine distance betweenp¯en\\bar\{p\}\_\{\\text\{en\}\}andp¯ℓ\\bar\{p\}\_\{\\ell\}\(Eq\.[3](https://arxiv.org/html/2606.26466#S2.E3)\)\. Embedding\-space alignment wins by a wide margin: in probability space accuracy drops 18\.0 points and CLC 30\.9 points, falling below SFT and the base model\. The collapse is especially severe for CLC because equivalent reasoning across languages activates largely disjoint token subsets inℝ\|V\|\\mathbb\{R\}^\{\|V\|\}, so cosine distance gives little useful gradient\. The embedding projection, by contrast, maps these probabilities into a shared space where cross\-script equivalences are captured\. Alternative distance metrics such as KL or Jensen–Shannon divergence are left to future work\.

#### Pivot languageℓ∗\\ell^\{\*\}\.

Table[17](https://arxiv.org/html/2606.26466#A6.T17)\(c\) reports per\-language results for Qwen3\-4B trained with different pivot languages\. All other hyperparameters are fixed atλ=0\.50\\lambda\{=\}0\.50andτ=1\.0\\tau\{=\}1\.0\. English yields the best aggregate performance \(78\.27% accuracy, 75\.61% CLC\), followed by Chinese \(74\.50%, 70\.97%\) and French \(72\.87%, 66\.67%\)\. The performance gap is concentrated in low\-resource languages, where Swahili drops from 53\.48% \(En\) to 40\.56% \(Zh\) and 36\.80% \(Fr\), consistent with the hypothesis that transfer effectiveness depends on the quality of the pivot’s pre\-trained representations\. English, as the dominant language in pre\-training corpora, provides the strongest anchor for alignment\.

Table 16:Per\-language accuracy, average accuracy, and cross\-lingual consistency \(%\) across four benchmarks for Qwen3\-4B SOLAR fine\-tuned variants with differentλ\\lambda\.Benchmarkλ\\lambdaEnFrJaSwTeThZhAvg\.\(↑\\uparrow\)CLC\(↑\\uparrow\)MGSM0\.2592\.5685\.9283\.4452\.8865\.6880\.1689\.6078\.6175\.730\.5093\.3286\.3281\.0053\.4865\.3680\.5687\.8478\.2775\.610\.7592\.4086\.0076\.7253\.5262\.7274\.2485\.2075\.8373\.731\.0087\.5283\.7672\.3242\.5653\.6864\.8080\.0869\.2563\.67AIME 20240\.2545\.0051\.3332\.3330\.6727\.3335\.0040\.3337\.4327\.830\.5045\.3350\.6734\.3332\.6729\.6738\.6738\.3338\.5229\.670\.7542\.3346\.3331\.6724\.3317\.3331\.0033\.6732\.3824\.441\.0040\.3341\.6730\.3310\.003\.0019\.3332\.3325\.2819\.22AIME 20250\.2535\.0036\.0029\.3326\.3321\.3329\.0035\.6730\.3823\.810\.5035\.3337\.3326\.6726\.3323\.3331\.6736\.0030\.9524\.510\.7532\.6731\.6722\.3320\.6716\.0026\.0032\.6726\.0020\.061\.0030\.0028\.3320\.679\.007\.0021\.6725\.6720\.3315\.83GPQA0\.2558\.6953\.4333\.1335\.3532\.1245\.8650\.5144\.1635\.470\.5058\.0853\.8434\.2439\.0933\.2344\.9553\.2345\.2436\.260\.7555\.2549\.9028\.2834\.4425\.5637\.9850\.2040\.2331\.871\.0050\.5139\.8020\.6118\.6914\.0427\.2733\.2329\.1617\.02Table 17:Per\-language results for the secondary ablations of SOLAR with Qwen3\-4B on MGSM: soft\-token temperatureτ\\tau, representation space, and pivot languageℓ∗\\ell^\{\*\}\.ConfigurationEnFrJaSwTeThZhAvg\.\(↑\\uparrow\)CLC\(↑\\uparrow\)\(a\) Soft\-token temperatureτ\\tau0\.689\.9075\.7670\.7243\.3614\.4862\.4866\.6460\.4857\.031\.093\.3286\.3281\.0053\.4865\.3680\.5687\.8478\.2775\.612\.091\.4479\.2879\.7656\.3267\.8474\.0886\.1676\.4173\.13\(b\) Representation spaceℝd\\mathbb\{R\}^\{d\}\(emb\.\)93\.3286\.3281\.0053\.4865\.3680\.5687\.8478\.2775\.61ℝ\|V\|\\mathbb\{R\}^\{\|V\|\}\(prob\.\)89\.2875\.0764\.6433\.4420\.8866\.5672\.0060\.2744\.76\(c\) Pivot languageℓ∗\\ell^\{\*\}English \(En\)93\.3286\.3281\.0053\.4865\.3680\.5687\.8478\.2775\.61Chinese \(Zh\)89\.6880\.8079\.8440\.5662\.4080\.0888\.1674\.5070\.97French \(Fr\)84\.2482\.7279\.6836\.8064\.0878\.4884\.0872\.8766\.67Table 18:Alignment\-target ablation for SOLAR with Qwen3\-4B and Qwen3\-8B\.MGSMAIME 2024AIME 2025GPQAAvg\.Alignment targetAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCAccuracyCLCQwen3\-4B\+\+SFT76\.2572\.2934\.7625\.1727\.9020\.4443\.7533\.1545\.6737\.76\+ Discrete76\.3773\.5035\.6326\.0628\.1221\.4143\.8633\.4746\.0038\.61\+ Final\-hidden76\.9773\.7935\.4725\.9127\.8720\.0544\.0534\.5146\.0938\.57\+ Untruncated77\.1974\.7237\.0327\.6327\.9322\.9344\.0434\.4646\.5539\.94\+ SOLAR \(k=30k\{=\}30\)78\.2775\.6138\.5229\.6730\.9524\.5145\.2436\.2648\.2541\.51Qwen3\-8B\+\+SFT86\.9084\.7165\.6758\.6758\.3850\.1363\.6455\.7368\.6562\.31\+ Discrete87\.2985\.4565\.0056\.8958\.8651\.0562\.9955\.0468\.5462\.11\+ Final\-hidden87\.0585\.0966\.5158\.8959\.0651\.1263\.4155\.6569\.0162\.69\+ Untruncated87\.9285\.7366\.9459\.0260\.0851\.9464\.0956\.1369\.7663\.21\+ SOLAR \(k=30k\{=\}30\)88\.2787\.2867\.5260\.6160\.1053\.0264\.9657\.2770\.2164\.55

#### Full alignment\-target results\.

Table[18](https://arxiv.org/html/2606.26466#A6.T18)extends the alignment\-target ablation of §[4\.4](https://arxiv.org/html/2606.26466#S4.SS4)to Qwen3\-8B alongside Qwen3\-4B\. At both scales SOLAR is the best target and a single hard token \(Discrete\) stays at the SFT level, while the intermediate targets \(Final\-hidden, Untruncated\) fall within about a point of one another and are not strictly ordered across benchmarks\. The consistent SOLAR\-best pattern across scales indicates that the soft mixture, rather than model size, drives the gain\.

### F\.2Representation analysis

#### Setup\.

We sample 50 instances per language for MGSM and GPQA and all 30 from each AIME subset\. We extract hidden states in two passes\. Pass 1 runs a standard forward pass, mean\-pools hidden states at each target layer, and builds soft tokens via Eq\.[2](https://arxiv.org/html/2606.26466#S2.E2)withτ=0\.6\\tau=0\.6andk=30k=30\. Pass 2 feeds these soft tokens back into the same model and extracts hidden states at the same layers\. We probe three depths: early \(layer 1\), middle \(∼\\sim50%\), and final \(layer−1\-1\), reportingΔ\\DeltaCLS andΔ\\DeltaSS relative to discrete tokens\. PositiveΔ\\DeltaCLS or negativeΔ\\DeltaSS indicates improved cross\-lingual mixing\.

#### Results\.

Tables[19](https://arxiv.org/html/2606.26466#A6.T19)–[30](https://arxiv.org/html/2606.26466#A6.T30)present the layer\-wise cross\-lingual alignment analysis for all 10 models across four benchmarks and 11 languages\. The inverted\-U pattern from §[4\.1](https://arxiv.org/html/2606.26466#S4.SS1)holds across all models and benchmarks: CLS peaks at middle layers and drops sharply at the final layer\. Soft thinking improves final\-layer CLS for most models and benchmarks, with magnitude varying by family: Gemma3 shows the largest gains \(Δ\\DeltaCLS up to \+0\.68\), Qwen3 improves more modestly, and DeepSeek\-R1\-Distill shows the smallest effect, with small negativeΔ\\DeltaCLS for DeepSeek\-R1\-Distill\-Llama\-8B on AIME 2024/2025 and GPQA\. Table[31](https://arxiv.org/html/2606.26466#A6.T31)reports CLS and SS at three depths for Qwen3\-4B, Qwen3\-8B, and their SFT and SOLAR variants, extending the final\-layer summary in §[4\.1](https://arxiv.org/html/2606.26466#S4.SS1)to early and middle layers\. The 8B results corroborate the 4B patterns, with gains concentrated at the final layer: SOLAR raises final\-layer discrete CLS from 0\.23 to 0\.76 \(Δ\\DeltaCLS = \+0\.53\), and soft thinking lifts it further to 0\.87 \(\+0\.64\)\. For 4B, SOLAR’s discrete gain \(\+0\.36\) exceeds its cumulative soft\-thinking gain \(\+0\.28\), so SOLAR has largely internalized what soft thinking provides\. On 8B, soft thinking still adds on top of SOLAR \(\+0\.64 vs\. \+0\.53\), suggesting the larger model retains capacity for further inference\-time alignment\. Early and middle layers change minimally at both scales, confirming the effect is specific to the final layer\.

Table 19:Cross\-lingual alignment on MGSM, discrete vs\. soft thinking \(Qwen3\)\.Qwen3\-4B\-BaseQwen3\-4BQwen3\-8B\-BaseQwen3\-8BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.840\.980\.280\.780\.940\.140\.850\.940\.730\.820\.970\.23Soft\-CLS\(↑\\uparrow\)0\.830\.960\.430\.780\.920\.240\.850\.910\.790\.830\.940\.36Δ\\DeltaCLS\-0\.01\-0\.02\+0\.15\+0\.00\-0\.02\+0\.10\+0\.00\-0\.03\+0\.06\+0\.01\-0\.03\+0\.13Disc\-SS\(↓\\downarrow\)0\.530\.050\.840\.580\.230\.890\.500\.140\.820\.530\.150\.87Soft\-SS\(↓\\downarrow\)0\.600\.130\.830\.670\.320\.880\.570\.100\.770\.600\.270\.83Δ\\DeltaSS\+0\.07\+0\.08\-0\.01\+0\.09\+0\.09\-0\.01\+0\.07\-0\.04\-0\.05\+0\.07\+0\.12\-0\.04

Table 20:Cross\-lingual alignment on AIME 2024, discrete vs\. soft thinking \(Qwen3\)\.Qwen3\-4B\-BaseQwen3\-4BQwen3\-8B\-BaseQwen3\-8BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.880\.980\.420\.870\.960\.390\.890\.950\.850\.890\.980\.45Soft\-CLS\(↑\\uparrow\)0\.870\.960\.580\.860\.950\.510\.890\.910\.900\.900\.950\.60Δ\\DeltaCLS\-0\.01\-0\.02\+0\.16\-0\.01\-0\.01\+0\.12\+0\.00\-0\.04\+0\.05\+0\.01\-0\.03\+0\.15Disc\-SS\(↓\\downarrow\)0\.42\-0\.050\.700\.420\.010\.700\.390\.040\.480\.38\-0\.060\.65Soft\-SS\(↓\\downarrow\)0\.450\.010\.680\.460\.080\.670\.410\.010\.480\.40\-0\.020\.62Δ\\DeltaSS\+0\.03\+0\.06\-0\.02\+0\.04\+0\.07\-0\.03\+0\.02\-0\.03\+0\.00\+0\.02\+0\.04\-0\.03

Table 21:Cross\-lingual alignment on AIME 2025, discrete vs\. soft thinking \(Qwen3\)\.Qwen3\-4B\-BaseQwen3\-4BQwen3\-8B\-BaseQwen3\-8BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.880\.980\.450\.880\.960\.430\.900\.950\.870\.900\.970\.48Soft\-CLS\(↑\\uparrow\)0\.880\.960\.590\.870\.950\.540\.900\.920\.910\.900\.940\.63Δ\\DeltaCLS\+0\.00\-0\.02\+0\.14\-0\.01\-0\.01\+0\.11\+0\.00\-0\.03\+0\.04\+0\.00\-0\.03\+0\.15Disc\-SS\(↓\\downarrow\)0\.40\-0\.130\.690\.410\.000\.700\.380\.020\.500\.38\-0\.090\.65Soft\-SS\(↓\\downarrow\)0\.43\-0\.040\.680\.460\.080\.660\.41\-0\.010\.500\.40\-0\.040\.61Δ\\DeltaSS\+0\.03\+0\.09\-0\.01\+0\.05\+0\.08\-0\.04\+0\.03\-0\.03\+0\.00\+0\.02\+0\.05\-0\.04

Table 22:Cross\-lingual alignment on GPQA, discrete vs\. soft thinking \(Qwen3\)\.Qwen3\-4B\-BaseQwen3\-4BQwen3\-8B\-BaseQwen3\-8BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.880\.970\.390\.870\.950\.370\.890\.950\.830\.900\.970\.37Soft\-CLS\(↑\\uparrow\)0\.870\.960\.540\.860\.940\.460\.890\.920\.880\.900\.940\.54Δ\\DeltaCLS\-0\.01\-0\.01\+0\.15\-0\.01\-0\.01\+0\.09\+0\.00\-0\.03\+0\.05\+0\.00\-0\.03\+0\.17Disc\-SS\(↓\\downarrow\)0\.47\-0\.000\.720\.480\.070\.730\.440\.080\.600\.44\-0\.010\.70Soft\-SS\(↓\\downarrow\)0\.510\.060\.710\.530\.150\.720\.470\.120\.620\.460\.060\.66Δ\\DeltaSS\+0\.04\+0\.06\-0\.01\+0\.05\+0\.08\-0\.01\+0\.03\+0\.04\+0\.02\+0\.02\+0\.07\-0\.04

Table 23:Cross\-lingual alignment on MGSM, discrete vs\. soft thinking \(Gemma3\)\.Gemma3\-1BGemma3\-4BGemma3\-12BGemma3\-27BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.981\.000\.270\.991\.000\.280\.991\.000\.250\.991\.000\.30Soft\-CLS\(↑\\uparrow\)0\.961\.000\.750\.991\.000\.900\.931\.000\.930\.951\.000\.90Δ\\DeltaCLS\-0\.02\+0\.00\+0\.48\+0\.00\+0\.00\+0\.62\-0\.06\+0\.00\+0\.68\-0\.04\+0\.00\+0\.60Disc\-SS\(↓\\downarrow\)0\.410\.230\.870\.260\.070\.840\.310\.210\.82\-0\.03\-0\.150\.82Soft\-SS\(↓\\downarrow\)0\.600\.260\.340\.550\.150\.220\.540\.080\.090\.560\.120\.13Δ\\DeltaSS\+0\.19\+0\.03\-0\.53\+0\.29\+0\.08\-0\.62\+0\.23\-0\.13\-0\.73\+0\.59\+0\.27\-0\.69

Table 24:Cross\-lingual alignment on AIME 2024, discrete vs\. soft thinking \(Gemma3\)\.Gemma3\-1BGemma3\-4BGemma3\-12BGemma3\-27BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.991\.000\.441\.001\.000\.460\.991\.000\.420\.991\.000\.46Soft\-CLS\(↑\\uparrow\)0\.981\.000\.830\.991\.000\.930\.961\.000\.940\.971\.000\.92Δ\\DeltaCLS\-0\.01\+0\.00\+0\.39\-0\.01\+0\.00\+0\.47\-0\.03\+0\.00\+0\.52\-0\.02\+0\.00\+0\.46Disc\-SS\(↓\\downarrow\)0\.190\.090\.660\.140\.010\.670\.130\.080\.67\-0\.09\-0\.190\.69Soft\-SS\(↓\\downarrow\)0\.260\.100\.140\.210\.040\.050\.27\-0\.010\.030\.240\.01\-0\.01Δ\\DeltaSS\+0\.07\+0\.01\-0\.52\+0\.07\+0\.03\-0\.62\+0\.14\-0\.09\-0\.64\+0\.33\+0\.20\-0\.70

Table 25:Cross\-lingual alignment on AIME 2025, discrete vs\. soft thinking \(Gemma3\)\.Gemma3\-1BGemma3\-4BGemma3\-12BGemma3\-27BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.991\.000\.461\.001\.000\.480\.991\.000\.430\.991\.000\.48Soft\-CLS\(↑\\uparrow\)0\.981\.000\.850\.991\.000\.930\.961\.000\.930\.971\.000\.92Δ\\DeltaCLS\-0\.01\+0\.00\+0\.39\-0\.01\+0\.00\+0\.45\-0\.03\+0\.00\+0\.50\-0\.02\+0\.00\+0\.44Disc\-SS\(↓\\downarrow\)0\.150\.080\.650\.120\.000\.680\.100\.080\.69\-0\.10\-0\.160\.70Soft\-SS\(↓\\downarrow\)0\.250\.070\.100\.180\.030\.020\.27\-0\.03\-0\.040\.23\-0\.00\-0\.00Δ\\DeltaSS\+0\.10\-0\.01\-0\.55\+0\.06\+0\.03\-0\.66\+0\.17\-0\.11\-0\.73\+0\.33\+0\.16\-0\.70

Table 26:Cross\-lingual alignment on GPQA, discrete vs\. soft thinking \(Gemma3\)\.Gemma3\-1BGemma3\-4BGemma3\-12BGemma3\-27BMetricEarlyMidFinalEarlyMidFinalEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.991\.000\.411\.001\.000\.400\.991\.000\.340\.991\.000\.37Soft\-CLS\(↑\\uparrow\)0\.981\.000\.780\.991\.000\.920\.951\.000\.920\.971\.000\.89Δ\\DeltaCLS\-0\.01\+0\.00\+0\.37\-0\.01\+0\.00\+0\.52\-0\.04\+0\.00\+0\.58\-0\.02\+0\.00\+0\.52Disc\-SS\(↓\\downarrow\)0\.250\.070\.730\.160\.010\.740\.140\.070\.75\-0\.03\-0\.150\.77Soft\-SS\(↓\\downarrow\)0\.360\.220\.310\.300\.160\.190\.350\.040\.060\.350\.060\.03Δ\\DeltaSS\+0\.11\+0\.15\-0\.42\+0\.14\+0\.15\-0\.55\+0\.21\-0\.03\-0\.69\+0\.38\+0\.21\-0\.74

Table 27:Cross\-lingual alignment on MGSM, discrete vs\. soft thinking \(DeepSeek\-R1\-Distill\)\.DeepSeek\-R1\-Distill\-Qwen\-7BDeepSeek\-R1\-Distill\-Llama\-8BMetricEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.741\.000\.240\.800\.880\.20Soft\-CLS\(↑\\uparrow\)0\.730\.970\.250\.850\.900\.21Δ\\DeltaCLS\-0\.01\-0\.03\+0\.01\+0\.05\+0\.02\+0\.01Disc\-SS\(↓\\downarrow\)0\.550\.110\.830\.480\.160\.77Soft\-SS\(↓\\downarrow\)0\.590\.200\.820\.500\.260\.80Δ\\DeltaSS\+0\.04\+0\.09\-0\.01\+0\.02\+0\.10\+0\.03Table 28:Cross\-lingual alignment on AIME 2024, discrete vs\. soft thinking \(DeepSeek\-R1\-Distill\)\.DeepSeek\-R1\-Distill\-Qwen\-7BDeepSeek\-R1\-Distill\-Llama\-8BMetricEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.811\.000\.330\.860\.890\.36Soft\-CLS\(↑\\uparrow\)0\.810\.970\.340\.870\.900\.34Δ\\DeltaCLS\+0\.00\-0\.03\+0\.01\+0\.01\+0\.01\-0\.02Disc\-SS\(↓\\downarrow\)0\.390\.010\.600\.330\.080\.66Soft\-SS\(↓\\downarrow\)0\.390\.070\.640\.320\.130\.66Δ\\DeltaSS\-0\.00\+0\.06\+0\.04\-0\.01\+0\.05\+0\.00Table 29:Cross\-lingual alignment on AIME 2025, discrete vs\. soft thinking \(DeepSeek\-R1\-Distill\)\.DeepSeek\-R1\-Distill\-Qwen\-7BDeepSeek\-R1\-Distill\-Llama\-8BMetricEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.830\.990\.350\.870\.890\.39Soft\-CLS\(↑\\uparrow\)0\.820\.970\.350\.880\.890\.37Δ\\DeltaCLS\-0\.01\-0\.02\+0\.00\+0\.01\+0\.00\-0\.02Disc\-SS\(↓\\downarrow\)0\.38\-0\.090\.610\.290\.040\.66Soft\-SS\(↓\\downarrow\)0\.370\.030\.640\.280\.060\.65Δ\\DeltaSS\-0\.01\+0\.12\+0\.03\-0\.01\+0\.02\-0\.01Table 30:Cross\-lingual alignment on GPQA, discrete vs\. soft thinking \(DeepSeek\-R1\-Distill\)\.DeepSeek\-R1\-Distill\-Qwen\-7BDeepSeek\-R1\-Distill\-Llama\-8BMetricEarlyMidFinalEarlyMidFinalDisc\-CLS\(↑\\uparrow\)0\.810\.990\.290\.860\.880\.32Soft\-CLS\(↑\\uparrow\)0\.800\.960\.300\.860\.880\.31Δ\\DeltaCLS\-0\.01\-0\.03\+0\.01\+0\.00\+0\.00\-0\.01Disc\-SS\(↓\\downarrow\)0\.46\-0\.020\.720\.400\.130\.69Soft\-SS\(↓\\downarrow\)0\.470\.110\.730\.370\.190\.72Δ\\DeltaSS\+0\.01\+0\.13\+0\.01\-0\.03\+0\.06\+0\.03Table 31:Cross\-lingual representation analysis across three layer depths of Qwen3\-4B/8B and their fine\-tuned variants\.Δ\\Deltais computed against each model’s base \(discrete\-thinking\) configuration\.ModelInferenceLayerCLS\(↑\\uparrow\)SS\(↓\\downarrow\)Δ\\DeltaCLSΔ\\DeltaSSQwen3\-4BDiscrete ThinkingEarly0\.780\.580\.000\.00Mid0\.940\.230\.000\.00Final0\.140\.890\.000\.00Soft ThinkingEarly0\.780\.67\+0\.00\+0\.09Mid0\.920\.32\-0\.02\+0\.09Final0\.240\.88\+0\.10\-0\.01\+ SFTDiscrete ThinkingEarly0\.800\.57\+0\.02\-0\.01Mid0\.950\.17\+0\.01\-0\.06Final0\.300\.86\+0\.16\-0\.03Soft ThinkingEarly0\.800\.66\+0\.02\+0\.08Mid0\.930\.26\-0\.01\+0\.03Final0\.400\.85\+0\.26\-0\.04\+ SOLARDiscrete ThinkingEarly0\.800\.57\+0\.02\-0\.01Mid0\.950\.21\+0\.01\-0\.02Final0\.500\.77\+0\.36\-0\.12Soft ThinkingEarly0\.780\.65\+0\.00\+0\.07Mid0\.900\.42\-0\.04\+0\.19Final0\.420\.81\+0\.28\-0\.08Qwen3\-8BDiscrete ThinkingEarly0\.820\.530\.000\.00Mid0\.970\.150\.000\.00Final0\.230\.870\.000\.00Soft ThinkingEarly0\.830\.60\+0\.01\+0\.07Mid0\.940\.27\-0\.03\+0\.12Final0\.360\.83\+0\.13\-0\.04\+ SFTDiscrete ThinkingEarly0\.820\.53\+0\.00\+0\.00Mid0\.970\.19\+0\.00\+0\.04Final0\.500\.86\+0\.27\-0\.01Soft ThinkingEarly0\.810\.59\-0\.01\+0\.06Mid0\.940\.22\-0\.03\+0\.07Final0\.560\.86\+0\.33\-0\.01\+ SOLARDiscrete ThinkingEarly0\.850\.52\+0\.03\-0\.01Mid0\.970\.09\+0\.00\-0\.06Final0\.760\.77\+0\.53\-0\.10Soft ThinkingEarly0\.850\.62\+0\.03\+0\.09Mid0\.930\.19\-0\.04\+0\.04Final0\.870\.76\+0\.64\-0\.11

### F\.3Behavioral analysis

We examine whether fine\-tuning changes the model’s reasoning language when the prompt is non\-English\. We evaluate Qwen3\-4B on MGSM across four non\-Latin\-script languages spanning high\-, mid\-, and low\-resource settings \(Zh, Ja, Th, Te\) and analyze both the*token*\-level \(generated tokens\) and the*distribution*\-level \(top\-30 tokens in the output distribution during decoding\)\. To quantify language use in reasoning traces and top\-kkdistributions, we classify every token into one of three categories by Unicode code\-point range: \(1\)target\-script: characters belonging to the prompt language’s native writing system \(CJK Unified IdeographsU\+4E00\-\-9FFFand extensions for Chinese; CJK plus HiraganaU\+3040\-\-309Fand KatakanaU\+30A0\-\-30FFfor Japanese; ThaiU\+0E00\-\-0E7F; TeluguU\+0C00\-\-0C7F\); \(2\)Latin: characters whose Unicode name containslatin, a proxy for English; and \(3\)neutral: digits, punctuation, mathematical symbols, and whitespace \(Unicode categoriesN,P,S, andZ\), which are script\-independent and shared across languages\. We average over five MGSM runs\. Thetarget\-script ratiois defined astarget/\(target\+Latin\)\\text\{target\}/\(\\text\{target\}\+\\text\{Latin\}\), which excludes neutral tokens from the comparison\.

#### Token\-level composition\.

Tables[32](https://arxiv.org/html/2606.26466#A6.T32)–[35](https://arxiv.org/html/2606.26466#A6.T35)report the target\-script ratio of generated reasoning traces, overall and by answer correctness\. On MGSM, all 4B variants reason almost entirely in the target script \(96–99%, average correct/incorrectΔ<0\.5%\\Delta<0\.5\\%\)\. The 8B base splits sharply: Chinese stays high \(99\.15%\), but Japanese, Thai, and Telugu fall below 5%, with the model reasoning in English\. SFT and SOLAR both recover all four above 97%\. On harder benchmarks \(AIME 2024, AIME 2025, GPQA\), target\-script ratios drop for all models\. The 4B base declines the most \(75–80% average\)\. Fine\-tuned models stay more robust \(roughly 77–99%, Telugu highest\)\. The 8B base retains its English\-thinking pattern \(Ja/Th/Te below 1\.1%\), while SFT and SOLAR again recover to 86–89%\.

#### Top\-kkcomposition\.

We perform teacher\-forced decoding and classify the top\-30 tokens at each position as*target\-script*,*Latin*, or*neutral*\. Figure[5](https://arxiv.org/html/2606.26466#S4.F5)\(top row\) shows the composition breakdown and Table[36](https://arxiv.org/html/2606.26466#A6.T36)the target\-script ratio, revealing two findings\. First, the 8B base has a stronger English preference than the 4B base\. SFT and SOLAR raise its MGSM average from 38\.05% to above 87%\. Second, the two methods shift this preference differently: SFT increases target\-language token density, while SOLAR suppresses Latin competition without proportionally adding target tokens\. The contrast is clearest for Telugu, where SOLAR reaches 98\.37% \(4B\) while SFT barely moves the base ratio \(65\.21% vs\. 65\.37%\), consistent with SOLAR’s alignment objective reducing cross\-lingual hedging\. These patterns extend to the harder benchmarks\. The 4B base now shows the distribution\-level collapse previously seen only in the 8B base on MGSM\. Fine\-tuning again recovers most of the loss, and SOLAR keeps the highest Telugu ratio on every benchmark\.

#### Top\-ppcomposition\.

Because the fixed budget \(k=30k\{=\}30\) may include low\-probability tokens the model would never sample, we repeat the composition analysis under nucleus \(top\-pp\) sampling withp=0\.9p\{=\}0\.9, retaining the minimal token set whose cumulative probability exceeds the threshold\. Table[36](https://arxiv.org/html/2606.26466#A6.T36)reports the top\-pptarget\-script ratio, and Figure[5](https://arxiv.org/html/2606.26466#S4.F5)\(bottom row\) the full breakdown\. The 8B base’s English dominance persists under top\-pp\(below 33% for Japanese, Thai, and Telugu\), confirming that the top\-kkpreference is not an artifact of low\-probability tail tokens\. SFT and SOLAR again raise its average from 41\.37% to above 97%\. After fine\-tuning, the primary generation intent is strongly target\-language, and the residual cross\-lingual competition sits in the low\-probability tail\. The same trends hold on AIME 2024, AIME 2025, and GPQA: base models retain their collapse while trained models reach high top\-ppratios\. The top\-pp/top\-kkgap narrows on the harder benchmarks, indicating that on hard problems English alternatives sit higher in the probability mass\. SOLAR attains the highest Telugu ratios throughout \(99\.34% / 99\.47% top\-ppfor 4B / 8B\)\.

Table 32:Target\-script ratio \(%\) on MGSM reasoning traces\.ModelZhJaThTeAvg\.Qwen3\-4BOverall98\.5699\.2499\.4198\.0198\.81Correct98\.6499\.2199\.3997\.6698\.72Incorrect98\.0199\.3899\.5198\.6998\.90\+ SFTOverall97\.7797\.7198\.5499\.5398\.39Correct97\.7997\.4998\.4599\.4898\.30Incorrect97\.5298\.4598\.8999\.6298\.63\+ SOLAROverall97\.2996\.9898\.4699\.7798\.13Correct97\.6897\.1498\.2999\.7698\.22Incorrect95\.7696\.7798\.8099\.7897\.77Qwen3\-8BOverall99\.152\.264\.174\.1427\.43Correct99\.232\.103\.973\.8127\.28Incorrect98\.523\.135\.805\.4728\.23\+ SFTOverall97\.7697\.6498\.5099\.5298\.36Correct98\.0497\.6698\.5299\.5998\.45Incorrect95\.7597\.6098\.4399\.3397\.77\+ SOLAROverall97\.6397\.4897\.7599\.8098\.16Correct97\.9797\.6897\.9799\.8198\.36Incorrect97\.0997\.0497\.2999\.7997\.80Table 33:Target\-script ratio \(%\) on AIME 2024 reasoning traces\.ModelZhJaThTeAvg\.Qwen3\-4BOverall60\.9680\.5890\.3586\.8779\.69Correct66\.5978\.9387\.9269\.0375\.62Incorrect56\.7181\.4891\.1188\.7779\.52\+ SFTOverall80\.9888\.3292\.4595\.2789\.26Correct77\.5485\.1592\.1491\.5386\.59Incorrect83\.3990\.2892\.5896\.2790\.63\+ SOLAROverall79\.9689\.4194\.2097\.5190\.27Correct73\.0178\.6991\.5692\.1283\.84Incorrect82\.3990\.7794\.5797\.7791\.37Qwen3\-8BOverall80\.410\.411\.010\.9820\.70Correct80\.040\.401\.041\.1220\.65Incorrect81\.330\.450\.910\.8820\.89\+ SFTOverall79\.6286\.1191\.9294\.6888\.08Correct79\.3584\.3491\.4791\.9086\.77Incorrect80\.0487\.9992\.2695\.4488\.94\+ SOLAROverall77\.4088\.9388\.4398\.0988\.21Correct73\.9686\.1489\.0994\.8186\.00Incorrect77\.6689\.3288\.2698\.2288\.37Table 34:Target\-script ratio \(%\) on AIME 2025 reasoning traces\.ModelZhJaThTeAvg\.Qwen3\-4BOverall57\.9581\.2491\.9189\.9580\.26Correct63\.5780\.1788\.1111\.7460\.90Incorrect55\.5581\.8393\.0792\.4180\.72\+ SFTOverall83\.3888\.6293\.9895\.3290\.32Correct80\.1786\.2792\.2792\.7387\.86Incorrect85\.1389\.5794\.8495\.8491\.34\+ SOLAROverall81\.1489\.2495\.2697\.2090\.71Correct79\.5883\.6793\.4497\.6088\.57Incorrect81\.9789\.9595\.6697\.2091\.19Qwen3\-8BOverall81\.690\.200\.540\.4220\.71Correct80\.780\.100\.290\.4020\.39Incorrect83\.030\.360\.870\.4321\.17\+ SFTOverall80\.8886\.9592\.7994\.4288\.76Correct79\.6986\.0091\.7492\.4187\.46Incorrect81\.8487\.4393\.4494\.9589\.41\+ SOLAROverall80\.2388\.2788\.2798\.0488\.70Correct78\.8088\.2590\.3095\.2388\.15Incorrect80\.6888\.3387\.9698\.0988\.77Table 35:Target\-script ratio \(%\) on GPQA reasoning traces\.ModelZhJaThTeAvg\.Qwen3\-4BOverall70\.3178\.3082\.4870\.0875\.29Correct69\.8980\.1380\.9839\.4967\.62Incorrect70\.5777\.9783\.1275\.2276\.72\+ SFTOverall83\.8190\.0489\.7596\.0289\.90Correct82\.2687\.4589\.1294\.5988\.36Incorrect85\.2991\.5690\.2296\.6490\.93\+ SOLAROverall83\.1689\.0790\.7598\.1790\.28Correct80\.7387\.7490\.3196\.8688\.91Incorrect84\.2689\.3390\.8698\.3490\.70Qwen3\-8BOverall85\.200\.310\.670\.3721\.64Correct83\.950\.260\.720\.3421\.32Incorrect86\.550\.370\.620\.4021\.98\+ SFTOverall83\.8489\.3089\.6395\.5589\.58Correct82\.1887\.3689\.4794\.8588\.47Incorrect85\.5291\.0789\.7495\.9490\.57\+ SOLAROverall77\.5188\.0282\.9698\.8486\.83Correct77\.3086\.1887\.5798\.6487\.42Incorrect77\.5988\.5281\.4898\.8886\.62Table 36:Target\-script ratio \(%\), top\-kk\(k=30k\{=\}30\) and top\-pp\(p=0\.9p\{=\}0\.9\) token selection on MGSM, AIME 2024, AIME 2025, and GPQA\.Top\-kk\(k=30k\{=\}30\)Top\-pp\(p=0\.9p\{=\}0\.9\)BenchmarkModelZhJaThTeAvg\.ZhJaThTeAvg\.MGSMQwen3\-4B82\.2678\.8787\.7065\.3778\.5594\.5795\.8497\.2294\.3795\.50\+ SFT91\.2992\.2691\.9165\.2185\.1795\.6997\.3598\.4198\.7597\.55\+ SOLAR79\.3481\.1491\.2198\.3787\.5195\.7597\.1698\.4699\.3497\.68Qwen3\-8B92\.4919\.7327\.9911\.9738\.0597\.9921\.7532\.5913\.1541\.37\+ SFT93\.0993\.0994\.4169\.0587\.4196\.2197\.1498\.4798\.7897\.65\+ SOLAR82\.8377\.7991\.7598\.9987\.8495\.6896\.0597\.3899\.4797\.14AIME 2024Qwen3\-4B83\.583\.801\.301\.7222\.6083\.302\.361\.661\.5822\.22\+ SFT76\.7881\.2586\.4555\.7475\.0578\.6484\.6293\.0888\.0186\.09\+ SOLAR67\.1374\.7388\.3495\.2681\.3678\.1087\.6391\.4090\.6186\.94Qwen3\-8B84\.173\.032\.303\.0123\.1382\.752\.382\.913\.0922\.78\+ SFT81\.0284\.8789\.1160\.6278\.9180\.2786\.9392\.0389\.2687\.12\+ SOLAR67\.4867\.9188\.7898\.0280\.5578\.8184\.4991\.4292\.6086\.83AIME 2025Qwen3\-4B84\.553\.671\.161\.1322\.6384\.591\.681\.530\.7522\.14\+ SFT76\.8682\.6387\.0054\.1475\.1676\.7986\.1693\.1387\.7585\.96\+ SOLAR66\.0772\.8487\.8494\.0680\.2076\.9085\.0091\.6188\.2485\.44Qwen3\-8B84\.173\.431\.360\.7922\.4483\.521\.891\.660\.9922\.02\+ SFT82\.3685\.4990\.1459\.0379\.2681\.6687\.4992\.9787\.7387\.46\+ SOLAR65\.7669\.2785\.8297\.3479\.5577\.2884\.6289\.6792\.0385\.90GPQAQwen3\-4B86\.435\.805\.981\.1824\.8587\.345\.106\.420\.9024\.94\+ SFT76\.2482\.9386\.1759\.5576\.2275\.7185\.1790\.9291\.5885\.84\+ SOLAR69\.6379\.0689\.2197\.9583\.9682\.9590\.4992\.2995\.3890\.28Qwen3\-8B86\.885\.553\.782\.2024\.6186\.815\.914\.251\.9524\.73\+ SFT86\.9487\.3089\.6863\.6981\.9086\.7788\.8492\.6492\.8290\.27\+ SOLAR63\.5467\.3483\.0399\.0278\.2380\.2080\.4581\.3197\.0784\.76

### F\.4Case Study

We present two case studies where the SOLAR\-trained model answers correctly while the base Qwen3\-4B and the SFT\-only model both fail\. The examples show two qualitative advantages of SOLAR\. First, it produces shorter, more structured reasoning chains, avoiding the repeated dead ends of the base model\. Second, it reduces interpretation errors on multilingual mathematical expressions, translating symbolic constraints in non\-English problem statements into geometric reasoning\.

#### AIME 2025 Q28 \(Chinese\)\.

Figure[6](https://arxiv.org/html/2606.26466#A6.F6)shows the full reasoning traces\. This geometry problem asks for the area of a quadrilateral inscribed in a right triangle\. The base model correctly sets up the coordinate system and derives the main constraints but fails to solve the resulting nonlinear system, abandoning the derivation and guessing \(n=147n=147\)\. The SFT model introduces an unsupported symmetry assumption, incorrectly settingb=cb=c, which yields the wrong answer98​398\\sqrt\{3\}\. SOLAR correctly enforces the constraints, obtainsb=16​3b=16\\sqrt\{3\}andc=26c=26, computes the vertex coordinates, and applies the shoelace formula to reach the correct answern=104n=104\.

#### AIME 2025 Q7 \(Japanese\)\.

Figure[7](https://arxiv.org/html/2606.26466#A6.F7)shows the full reasoning traces\. This problem requires finding real values ofkkfor which a circle and a perpendicular bisector intersect exactly once\. A key challenge is correctly interpreting\|z−4−k\|\|z\-4\-k\|and\|z−3​i−k\|\|z\-3i\-k\|, which correspond to distances to\(4\+k,0\)\(4\+k,0\)and\(k,3\)\(k,3\)\. The base model misreads them as\(4,k\)\(4,k\)and\(0,k\)\(0,k\), placing both points ony=ky=kand producing a fixed bisectorx=2x=2, which leads it to conclude that the line cannot intersect the circle after repeatedly revisiting the same steps\. The SFT model makes a different parsing error, implicitly treatingkkask​iki, which yields incorrect reference points and the wrong resultm\+n=8m\+n=8\. SOLAR derives the correct bisector−8​x\+6​y\+8​k\+7=0\-8x\+6y\+8k\+7=0and solves\|8​k−73\|=50\|8k\-73\|=50, producing the correct resultm\+n=77m\+n=77\.

Case Study: AIME 2025 Q28 \(Chinese\)Question:设△​A​B​C\\triangle ABC为一个直角三角形，∠​A=90∘\\angle A=90^\{\\circ\}且B​C=38BC=38。在三角形内部存在点KK和LL，使得A​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14。四边形B​K​L​CBKLC的面积可以表示为n​3n\\sqrt\{3\}，其中nn是某个正整数。求nn。\(Let△​A​B​C\\triangle ABCbe a right triangle with∠​A=90∘\\angle A=90^\{\\circ\}andB​C=38BC=38\. There exist pointsKKandLLinside the triangle such thatA​K=A​L=B​K=C​L=K​L=14\.AK=AL=BK=CL=KL=14\.The area of the quadrilateralB​K​L​CBKLCcan be expressed asn​3n\\sqrt\{3\}for some positive integernn\. Findnn\.\)Ground Truth:104Qwen3\-4B× Incorrect \(Answer: 147\)我们已知：△​A​B​C\\triangle ABC是直角三角形，∠​A=90∘\\angle A=90^\{\\circ\}，斜边B​C=38BC=38。点KK和LL在三角形内部，满足A​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14，四边形B​K​L​CBKLC的面积为n​3n\\sqrt\{3\}，求nn。第一步：理解点KK和LL的位置。A​K=A​L=14AK=AL=14：点KK和LL到点AA的距离相等。B​K=14BK=14，C​L=14CL=14。K​L=14KL=14。所以，KK在以AA为圆心、14 为半径的圆上，也在以BB为圆心、14 为半径的圆上。同理，LL在以AA为圆心、14 为半径的圆上，也在以CC为圆心、14 为半径的圆上。KK是圆\(A,14\)\(A,14\)和圆\(B,14\)\(B,14\)的交点。LL是圆\(A,14\)\(A,14\)和圆\(C,14\)\(C,14\)的交点。又因为K​L=14KL=14，且K,LK,L都在以AA为圆心的圆上，所以KK和LL都在以AA为中心、半径 14 的圆上，且它们之间的距离是 14。第二步：建立坐标系。设直角在AA，令A=\(0,0\)A=\(0,0\)，B=\(b,0\)B=\(b,0\)，C=\(0,c\)C=\(0,c\)。则斜边B​C=38BC=38，由距离公式：B​C2=b2\+c2=382=1444BC^\{2\}=b^\{2\}\+c^\{2\}=38^\{2\}=1444。第三步：求点KK。点KK满足A​K=14AK=14（x2\+y2=196x^\{2\}\+y^\{2\}=196）和B​K=14BK=14（\(x−b\)2\+y2=196\(x\-b\)^\{2\}\+y^\{2\}=196）。两式相减：−2​b​x\+b2=0⇒x=b2\-2bx\+b^\{2\}=0\\Rightarrow x=\\frac\{b\}\{2\}。代入得y2=196−b24y^\{2\}=196\-\\frac\{b^\{2\}\}\{4\}。同理，对点LL：A​L=14AL=14和C​L=14CL=14相减得y=c2y=\\frac\{c\}\{2\}，x2=196−c24x^\{2\}=196\-\\frac\{c^\{2\}\}\{4\}。第四步：利用K​L=14KL=14。K=\(b2,yK\)K=\\left\(\\frac\{b\}\{2\},y\_\{K\}\\right\)，L=\(xL,c2\)L=\\left\(x\_\{L\},\\frac\{c\}\{2\}\\right\)。K​L2=\(b2−xL\)2\+\(yK−c2\)2=196KL^\{2\}=\\left\(\\frac\{b\}\{2\}\-x\_\{L\}\\right\)^\{2\}\+\\left\(y\_\{K\}\-\\frac\{c\}\{2\}\\right\)^\{2\}=196。第五步：考虑对称性。猜测b=cb=c，则2​b2=14442b^\{2\}=1444，b2=722b^\{2\}=722。验证：yK2=196−180\.5=15\.5y\_\{K\}^\{2\}=196\-180\.5=15\.5，xL2=15\.5x\_\{L\}^\{2\}=15\.5。K​L2=2​\(180\.5−15\.5\)2≈180\.46≠196KL^\{2\}=2\(\\sqrt\{180\.5\}\-\\sqrt\{15\.5\}\)^\{2\}\\approx 180\.46\\neq 196。对称情况b=cb=c不满足K​L=14KL=14。第六步：尝试几何解释。KK和LL都在以AA为中心、半径 14 的圆上，K​L=14KL=14，由余弦定理：142=142\+142−2⋅14⋅14⋅cos⁡θ14^\{2\}=14^\{2\}\+14^\{2\}\-2\\cdot 14\\cdot 14\\cdot\\cos\\theta，解得cos⁡θ=12\\cos\\theta=\\frac\{1\}\{2\}，θ=60∘\\theta=60^\{\\circ\}。所以∠​K​A​L=60∘\\angle KAL=60^\{\\circ\}。第七步：利用∠​K​A​L=60∘\\angle KAL=60^\{\\circ\}。A​K=A​L=14AK=AL=14，∠​K​A​L=60∘\\angle KAL=60^\{\\circ\}，所以△​A​K​L\\triangle AKL是等边三角形。第八步。向量A​K→⋅A​L→=14⋅14⋅cos⁡60∘=98\\vec\{AK\}\\cdot\\vec\{AL\}=14\\cdot 14\\cdot\\cos 60^\{\\circ\}=98，即b2​xL\+c2​yK=98\\frac\{b\}\{2\}x\_\{L\}\+\\frac\{c\}\{2\}y\_\{K\}=98，b​xL\+c​yK=196bx\_\{L\}\+cy\_\{K\}=196。代入xL=196−c24x\_\{L\}=\\sqrt\{196\-\\frac\{c^\{2\}\}\{4\}\}，yK=196−b24y\_\{K\}=\\sqrt\{196\-\\frac\{b^\{2\}\}\{4\}\}：b​196−c24\+c​196−b24=196b\\sqrt\{196\-\\frac\{c^\{2\}\}\{4\}\}\+c\\sqrt\{196\-\\frac\{b^\{2\}\}\{4\}\}=196。第九步至第十步。设b=2​xb=2x，c=2​yc=2y，得x​196−y2\+y​196−x2=98x\\sqrt\{196\-y^\{2\}\}\+y\\sqrt\{196\-x^\{2\}\}=98。尝试x=yx=y得u2−196​u\+2401=0u^\{2\}\-196u\+2401=0，判别式2881228812不是完全平方，所以b≠cb\\neq c。第十一步：放弃坐标，用面积公式。△​A​K​L\\triangle AKL面积=12⋅14⋅14⋅sin⁡60∘=49​3=\\frac\{1\}\{2\}\\cdot 14\\cdot 14\\cdot\\sin 60^\{\\circ\}=49\\sqrt\{3\}。Given the complexity and that the answer is to be in the formn​3n\\sqrt\{3\}, and from the only trigonometric part, likely n is 147 or 49 or 196\. From the triangle AKL, area is49​349\\sqrt\{3\}\. Perhaps the area of BKLC is147​3147\\sqrt\{3\}\. I think the intended answer is147\\boxed\{147\}\.English translation\.We know:△​A​B​C\\triangle ABCis a right triangle with∠​A=90∘\\angle A=90^\{\\circ\}and hypotenuseB​C=38BC=38\. PointsKKandLLlie inside the triangle withA​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14; the area of quadrilateralB​K​L​CBKLCisn​3n\\sqrt\{3\}, findnn\.Step 1: Understanding the positions ofKKandLL\.A​K=A​L=14AK=AL=14meansK,LK,Lare equidistant fromAA\. WithB​K=14BK=14,KKlies on both the circle\(A,14\)\(A,14\)and\(B,14\)\(B,14\); similarlyLLlies on\(A,14\)\(A,14\)and\(C,14\)\(C,14\)\. SinceK​L=14KL=14and both are on the circle of radius1414aroundAA, their mutual distance is1414\.Step 2: Set up coordinates\. Place the right angle atAA:A=\(0,0\)A=\(0,0\),B=\(b,0\)B=\(b,0\),C=\(0,c\)C=\(0,c\)\. Thenb2\+c2=382=1444b^\{2\}\+c^\{2\}=38^\{2\}=1444\.Step 3: FindKK\. FromA​K=B​K=14AK=BK=14, subtracting givesx=b/2x=b/2,y2=196−b2/4y^\{2\}=196\-b^\{2\}/4\. Similarly forLL:y=c/2y=c/2,x2=196−c2/4x^\{2\}=196\-c^\{2\}/4\.Step 4: UseK​L=14KL=14\.K=\(b/2,yK\)K=\(b/2,y\_\{K\}\),L=\(xL,c/2\)L=\(x\_\{L\},c/2\), so\(b/2−xL\)2\+\(yK−c/2\)2=196\(b/2\-x\_\{L\}\)^\{2\}\+\(y\_\{K\}\-c/2\)^\{2\}=196\.Step 5: Try symmetry\. Guessingb=cb=cgivesb2=722b^\{2\}=722, yieldingK​L2≈180\.46≠196KL^\{2\}\\approx 180\.46\\neq 196, sob=cb=cfails\.Step 6: Geometric interpretation\. SinceK,LK,Llie on the circle of radius1414aroundAAwithK​L=14KL=14, the law of cosines givescos⁡θ=1/2\\cos\\theta=1/2, so∠​K​A​L=60∘\\angle KAL=60^\{\\circ\}\.Step 7: Use∠​K​A​L=60∘\\angle KAL=60^\{\\circ\}\. WithA​K=A​L=14AK=AL=14,△​A​K​L\\triangle AKLis equilateral\.Step 8\.A​K→⋅A​L→=98\\vec\{AK\}\\cdot\\vec\{AL\}=98, sob​xL\+c​yK=196bx\_\{L\}\+cy\_\{K\}=196, givingb​196−c2/4\+c​196−b2/4=196b\\sqrt\{196\-c^\{2\}/4\}\+c\\sqrt\{196\-b^\{2\}/4\}=196\.Steps 9–10\. Substitutingb=2​xb=2x,c=2​yc=2yyieldsx​196−y2\+y​196−x2=98x\\sqrt\{196\-y^\{2\}\}\+y\\sqrt\{196\-x^\{2\}\}=98\. Tryingx=yx=ygivesu2−196​u\+2401=0u^\{2\}\-196u\+2401=0with non\-square discriminant2881228812, sob≠cb\\neq c\.Step 11: Abandon coordinates, use area formulas\. Area of△​A​K​L=12⋅14⋅14⋅sin⁡60∘=49​3\\triangle AKL=\\frac\{1\}\{2\}\\cdot 14\\cdot 14\\cdot\\sin 60^\{\\circ\}=49\\sqrt\{3\}\. Given the complexity and that the answer is to be in the formn​3n\\sqrt\{3\}, and from the only trigonometric part, likely n is 147 or 49 or 196\. From the triangle AKL, area is49​349\\sqrt\{3\}\. Perhaps the area of BKLC is147​3147\\sqrt\{3\}\. I think the intended answer is147\\boxed\{147\}\.SFT× Incorrect \(Answer: 98\)给定一个直角三角形△​A​B​C\\triangle ABC，其中∠​A=90∘\\angle A=90^\{\\circ\}且B​C=38BC=38。在三角形内部存在点KK和LL，使得A​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14。我们需要求出四边形B​K​L​CBKLC的面积，该面积表示为n​3n\\sqrt\{3\}并确定nn。1\. 坐标设置：将AA置于原点\(0,0\)\(0,0\)，BB置于\(b,0\)\(b,0\)，CC置于\(0,c\)\(0,c\)。根据勾股定理，b2\+c2=382=1444b^\{2\}\+c^\{2\}=38^\{2\}=1444。2\. 点KK和LL的坐标：点KK位于AA和BB的垂直平分线上，坐标为\(b/2,y\)\(b/2,y\)。点LL位于AA和CC的垂直平分线上，坐标为\(e,c/2\)\(e,c/2\)。3\. 方程求解：根据距离条件，我们得到以下方程：\(b/2\)2\+y2=196\(b/2\)^\{2\}\+y^\{2\}=196，e2\+\(c/2\)2=196e^\{2\}\+\(c/2\)^\{2\}=196，b​196−\(c2\)/4\+c​196−\(b2\)/4=196b\\sqrt\{196\-\(c^\{2\}\)/4\}\+c\\sqrt\{196\-\(b^\{2\}\)/4\}=196。通过代入和简化，我们发现bb和cc的关系使得三角形A​B​CABC成为一个等腰直角三角形，从而b=cb=c。4\. 面积计算：四边形B​K​L​CBKLC可以通过坐标几何计算面积。使用鞋带公式计算出98​398\\sqrt\{3\}。因此，nn的值是98\\boxed\{98\}。English translation\.Given a right triangle△​A​B​C\\triangle ABCwith∠​A=90∘\\angle A=90^\{\\circ\}andB​C=38BC=38\. Inside the triangle there exist pointsKKandLLwithA​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14\. We need to find the area ofB​K​L​CBKLC, written asn​3n\\sqrt\{3\}, and determinenn\.1\. Coordinate setup: PlaceAAat the origin\(0,0\)\(0,0\),BBat\(b,0\)\(b,0\),CCat\(0,c\)\(0,c\)\. By the Pythagorean theorem,b2\+c2=382=1444b^\{2\}\+c^\{2\}=38^\{2\}=1444\.2\. Coordinates ofKKandLL:KKlies on the perpendicular bisector ofA​BABat\(b/2,y\)\(b/2,y\);LLlies on the perpendicular bisector ofA​CACat\(e,c/2\)\(e,c/2\)\.3\. Solving the equations: The distance conditions give\(b/2\)2\+y2=196\(b/2\)^\{2\}\+y^\{2\}=196,e2\+\(c/2\)2=196e^\{2\}\+\(c/2\)^\{2\}=196, andb​196−c2/4\+c​196−b2/4=196b\\sqrt\{196\-c^\{2\}/4\}\+c\\sqrt\{196\-b^\{2\}/4\}=196\. Through substitution and simplification, the relationship betweenbbandcc\(incorrectly\) forces△​A​B​C\\triangle ABCto be an isosceles right triangle, sob=cb=c\.4\. Area computation: Using the shoelace formula on the coordinates ofB​K​L​CBKLCyields98​398\\sqrt\{3\}\. Thereforen=98n=\\boxed\{98\}\.SOLAR✓ Correct \(Answer: 104\)给定一个直角三角形△​A​B​C\\triangle ABC，其中∠​A=90∘\\angle A=90^\{\\circ\}，且B​C=38BC=38。在三角形内部存在点KK和LL，使得A​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14。我们需要求出四边形B​K​L​CBKLC的面积，其形式为n​3n\\sqrt\{3\}，并求出nn。1\. 坐标设置：将AA放在原点\(0,0\)\(0,0\)，BB放在\(b,0\)\(b,0\)，CC放在\(0,c\)\(0,c\)。根据勾股定理，b2\+c2=382=1444b^\{2\}\+c^\{2\}=38^\{2\}=1444。2\. 点 K 和 L 的坐标：点KK满足A​K=B​K=14AK=BK=14，其坐标为\(b2,196−b24\)\\left\(\\frac\{b\}\{2\},\\sqrt\{196\-\\frac\{b^\{2\}\}\{4\}\}\\right\)。点LL满足A​L=C​L=14AL=CL=14，其坐标为\(196−c24,c2\)\\left\(\\sqrt\{196\-\\frac\{c^\{2\}\}\{4\}\},\\frac\{c\}\{2\}\\right\)。3\. 求解bb和cc：使用K​L=14KL=14的条件，并解出方程组：b​196−c24\+c​196−b24=196b\\sqrt\{196\-\\frac\{c^\{2\}\}\{4\}\}\+c\\sqrt\{196\-\\frac\{b^\{2\}\}\{4\}\}=196，b2\+c2=1444b^\{2\}\+c^\{2\}=1444。解得b=16​3b=16\\sqrt\{3\}且c=26c=26。4\. 坐标计算：点KK的坐标：\(8​3,2\)\(8\\sqrt\{3\},2\)。点LL的坐标：\(3​3,13\)\(3\\sqrt\{3\},13\)。点BB的坐标：\(16​3,0\)\(16\\sqrt\{3\},0\)。点CC的坐标：\(0,26\)\(0,26\)。5\. 使用鞋带公式计算面积：顶点按顺序排列：B​\(16​3,0\)B\(16\\sqrt\{3\},0\)，K​\(8​3,2\)K\(8\\sqrt\{3\},2\)，L​\(3​3,13\)L\(3\\sqrt\{3\},13\)，C​\(0,26\)C\(0,26\)。使用鞋带公式计算面积，结果为104​3104\\sqrt\{3\}。因此，nn的值是104\\boxed\{104\}。English translation\.Given a right triangle△​A​B​C\\triangle ABCwith∠​A=90∘\\angle A=90^\{\\circ\}andB​C=38BC=38\. Inside the triangle there exist pointsKKandLLwithA​K=A​L=B​K=C​L=K​L=14AK=AL=BK=CL=KL=14\. We need to find the area ofB​K​L​CBKLC, of the formn​3n\\sqrt\{3\}, and determinenn\.1\. Coordinate setup: PlaceAAat the origin\(0,0\)\(0,0\),BBat\(b,0\)\(b,0\),CCat\(0,c\)\(0,c\)\. By the Pythagorean theorem,b2\+c2=382=1444b^\{2\}\+c^\{2\}=38^\{2\}=1444\.2\. Coordinates ofKKandLL:KKsatisfiesA​K=B​K=14AK=BK=14, givingK=\(b2,196−b24\)K=\\left\(\\frac\{b\}\{2\},\\sqrt\{196\-\\frac\{b^\{2\}\}\{4\}\}\\right\)\.LLsatisfiesA​L=C​L=14AL=CL=14, givingL=\(196−c24,c2\)L=\\left\(\\sqrt\{196\-\\frac\{c^\{2\}\}\{4\}\},\\frac\{c\}\{2\}\\right\)\.3\. Solving forbbandcc: UsingK​L=14KL=14and solving the systemb​196−c24\+c​196−b24=196b\\sqrt\{196\-\\frac\{c^\{2\}\}\{4\}\}\+c\\sqrt\{196\-\\frac\{b^\{2\}\}\{4\}\}=196,b2\+c2=1444b^\{2\}\+c^\{2\}=1444, we obtainb=16​3b=16\\sqrt\{3\}andc=26c=26\.4\. Coordinate computation:K=\(8​3,2\)K=\(8\\sqrt\{3\},2\),L=\(3​3,13\)L=\(3\\sqrt\{3\},13\),B=\(16​3,0\)B=\(16\\sqrt\{3\},0\),C=\(0,26\)C=\(0,26\)\.5\. Area via shoelace formula: Ordering the verticesB​\(16​3,0\)B\(16\\sqrt\{3\},0\),K​\(8​3,2\)K\(8\\sqrt\{3\},2\),L​\(3​3,13\)L\(3\\sqrt\{3\},13\),C​\(0,26\)C\(0,26\)and applying the shoelace formula yields an area of104​3104\\sqrt\{3\}\. Thereforen=104n=\\boxed\{104\}\.

Figure 6:Case study on AIME 2025 Q28 \(Chinese\)\.Case Study: AIME 2025 Q7 \(Japanese\)Question:kkを実数とし、系\|25\+20​i−z\|=5\|25\+20i\-z\|=5と\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|が正確に一つの複素数解zzを持つとします。すべての可能なkkの値の合計はmn\\frac\{m\}\{n\}の形で表され、ここでmmとnnは互いに素な正の整数です。m\+nm\+nを求めなさい。ここでi=−1i=\\sqrt\{\-1\}です。\(Letkkbe real numbers such that the system\|25\+20​i−z\|=5\|25\+20i\-z\|=5and\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|has exactly one complex solutionzz\. The sum of all possible values ofkkcan be written asmn\\frac\{m\}\{n\}, wheremmandnnare relatively prime positive integers\. Findm\+nm\+n\. Herei=−1i=\\sqrt\{\-1\}\.\)Ground Truth:77Qwen3\-4B× Incorrect \(Answer: 1\)問題を丁寧にステップごとに解いていきましょう。ステップ1：問題の理解与えられた条件は以下の2つ：1\.\|25\+20​i−z\|=5\|25\+20i\-z\|=52\.\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|この2つの条件が正確に一つの複素数解zzを持つとき、すべての可能なkkの値の合計を求め、その合計がmn\\frac\{m\}\{n\}の形で表され、mmとnnが互いに素なら、m\+nm\+nを求めよ。ステップ2：条件1の解釈条件1：\|25\+20​i−z\|=5\|25\+20i\-z\|=5これは、複素平面上で、zzが点25\+20​i25\+20iから距離 5 の点にある、つまり円を表します。つまり、zzは中心\(25,20\)\(25,20\)、半径 5 の円上にある。ステップ3：条件2の解釈条件2：\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|左辺はzzと点4\+k4\+kの距離、右辺はzzと点3​i\+k3i\+kの距離。この式は、zzが点4\+k4\+kと点3​i\+k3i\+kの等距離にある、つまりzzはその2点の垂直二等分線（またはその複素平面上の対応）上にある。この条件は、zzが2つの点の等距離にあるという、垂直二等分線を表します。ステップ4：両方の条件が「正確に一つの解」を持つつまり、円と直線（垂直二等分線）がちょうど1点で交わるという条件。これは、円と直線が接するという条件です。したがって、問題は：円\|z−\(25\+20​i\)\|=5\|z\-\(25\+20i\)\|=5と、点A=4\+kA=4\+k、点B=3​i\+kB=3i\+kの垂直二等分線が、ちょうど1点で交わるような実数kkをすべて求め、その和を求めよ。ステップ5：垂直二等分線の性質点A=4\+kA=4\+k、点B=3​i\+kB=3i\+kを複素平面上に見ると：AAの座標は\(4,k\)\(4,k\)、BBの座標は\(0,k\)\(0,k\)なぜなら：4\+k=4\+0​i\+k⇒\(4,k\)4\+k=4\+0i\+k\\Rightarrow\(4,k\)、3​i\+k=0\+3​i\+k⇒\(0,k\)3i\+k=0\+3i\+k\\Rightarrow\(0,k\)つまり、点AAと点BBは、同じyy値kkを持つ。したがって、2点\(4,k\)\(4,k\)と\(0,k\)\(0,k\)は、水平線y=ky=k上にある。この2点の中点は：\(4\+02,k\+k2\)=\(2,k\)\\left\(\\frac\{4\+0\}\{2\},\\frac\{k\+k\}\{2\}\\right\)=\(2,k\)2点の間のベクトルは\(−4,0\)\(\-4,0\)、つまり水平方向。よって、垂直二等分線は垂直方向（xx方向に垂直）で、つまりx=2x=2の直線。なぜなら、水平線の垂直二等分線は、その中点を通る垂直線。したがって、条件2の解は、直線x=2x=2上の点。ステップ6：円と直線の接点条件円：\|z−\(25\+20​i\)\|=5\|z\-\(25\+20i\)\|=5。これは、中心C=\(25,20\)C=\(25,20\)、半径r=5r=5の円。直線：x=2x=2この2つがちょうど1点で交わる（つまり接する）ためには、中心から直線までの距離が半径に等しい。中心\(25,20\)\(25,20\)、直線x=2x=2。距離は：\|25−2\|=23\|25\-2\|=23これは、距離が 23で、半径は 5。つまり、距離は半径よりずっと大きい。したがって、直線x=2x=2は円と交わらない（距離\>\>半径）。しかし、問題では「正確に一つの複素数解zz」を持つとある。これは、円と垂直二等分線がちょうど1点で交わるということ。しかし、上記では、垂直二等分線は常にx=2x=2で、中心からその距離は常に 23 であり、半径 5 よりも大きい。つまり、すべてのkkに対して、直線x=2x=2は円と交わらない。これは、問題の条件に反する。どこか間違えている？ステップ7：再確認 — 垂直二等分線は本当にx=2x=2か？点A=4\+kA=4\+k→\\rightarrow座標\(4,k\)\(4,k\)点B=3​i\+kB=3i\+k→\\rightarrow座標\(0,k\)\(0,k\)両方ともy=ky=kにあり、xx座標が44と0。中点：\(4\+02,k\+k2\)=\(2,k\)\\left\(\\frac\{4\+0\}\{2\},\\frac\{k\+k\}\{2\}\\right\)=\(2,k\)ベクトルA​B→=\(−4,0\)\\vec\{AB\}=\(\-4,0\)→\\rightarrow水平垂直方向はyy方向→\\rightarrow垂直二等分線はx=2x=2で、yyが自由。つまり、すべてのkkに対して、垂直二等分線はx=2x=2という直線。そして、この直線と円\|z−\(25\+20​i\)\|=5\|z\-\(25\+20i\)\|=5の距離は、常に\|25−2\|=23\|25\-2\|=23であり、半径 5 より大きい。したがって、交点は存在しない。しかし、問題は「正確に一つの複素数解zzを持つ」とある。これは矛盾。ステップ8：誤解の可能性条件2：\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|これは、zzが点4\+k4\+kと3​i\+k3i\+kの距離が等しいという条件。しかし、kkは実数なので、点4\+k4\+kは\(4,k\)\(4,k\)、点3​i\+k3i\+kは\(0,k\)\(0,k\)。両方がy=ky=kにあり、xx値が異なる。垂直二等分線は、中点\(2,k\)\(2,k\)を通って、A​BABの垂直方向。A​BABの方向は\(−4,0\)\(\-4,0\)→\\rightarrow水平。垂直方向は\(0,1\)\(0,1\)→\\rightarrow垂直。したがって、垂直二等分線は、x=2x=2で、yyは自由。つまり、すべてのkkに対して、垂直二等分線はx=2x=2。そして、この直線と円\(x−25\)2\+\(y−20\)2=25\(x\-25\)^\{2\}\+\(y\-20\)^\{2\}=25との距離は、常に23\>523\>5。よって、交点は存在しない。問題が「正確に一つの解を持つ」とあるが、これは矛盾。ステップ9：どこが間違っているか？点3​i\+k3i\+kは\(0,k\)\(0,k\)で、kkは実数。点4\+k4\+kは\(4,k\)\(4,k\)。両方ともy=ky=k。中点\(2,k\)\(2,k\)。垂直方向はyy方向。垂直二等分線はx=2x=2。これは正しい。円の中心\(25,20\)\(25,20\)、半径 5。距離から直線x=2x=2までの距離は2323。23\>523\>5→\\rightarrow交点なし。よって、すべてのkkに対して解が存在しない。しかし問題は「正確に一つの複素数解zzを持つ」としている。つまり、あるkkに対して、解がちょうど1つ。これは、円と垂直二等分線が接するということ。しかし、垂直二等分線はx=2x=2で、距離が 23 なので、それは常に円の外側にある。よって、接することはない。矛盾。ステップ10：誤解の再確認 — 複素数の操作点zzは複素数。条件2：\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|これは、zzが2つの点の距離が等しいという条件。しかし、kkは実数なので、点4\+k4\+kと3​i\+k3i\+kは、yy値がkkで、xx値が 4 と 0。中点\(2,k\)\(2,k\)。垂直二等分線はx=2x=2。これは正しい。円の中心\(25,20\)\(25,20\)、半径 5。距離\|25−2\|=23\|25\-2\|=23。23\>523\>5→\\rightarrow交点なし。ステップ11：問題の読み直し問題は：「系\|25\+20​i−z\|=5\|25\+20i\-z\|=5と\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|が正確に一つの複素数解zzを持つとします。」つまり、あるkkに対して、この2つの条件を同時に満たすzzがちょうど1つ。これは、円と垂直二等分線が接する条件。しかし、垂直二等分線はx=2x=2、円は中心\(25,20\)\(25,20\)、半径 5。距離23\>523\>5→\\rightarrow交点なし。したがって、接することはない。どこかの前提が間違っている？ステップ12：\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|は、zzが\(4,k\)\(4,k\)と\(0,k\)\(0,k\)の等距離にあるという意味で、垂直二等分線はx=2x=2で、yyは自由。これは正しい。しかし、kkが変化しても、この直線は常にx=2x=2。つまり、すべてのkkに対して、垂直二等分線はx=2x=2。円の中心は\(25,20\)\(25,20\)、半径 5。距離は常に 23。23\>523\>5→\\rightarrow交点なし。したがって、解が存在しない。問題が「正確に一つの解を持つ」としているなら、そのようなkkは存在しない。しかし、問題は「すべての可能なkkの値の合計」という形で尋ねており、答えはmn\\frac\{m\}\{n\}で、m\+nm\+nを求めよ。つまり、少なくとも1つのkkが存在するとされている。ステップ13：誤解の可能性 —\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|の意味zzが複素数、kkは実数。この式は、zzが2つの点の距離が等しいということ。しかし、点\(4,k\)\(4,k\)と\(0,k\)\(0,k\)は、y=ky=kにあり、その垂直二等分線はx=2x=2。しかし、もしkkが変化しても、その直線はx=2x=2で、yyは自由。したがって、すべてのkkで、垂直二等分線はx=2x=2。円は固定。距離は固定。ステップ14：もしかすると、円もkkに依存？いいえ。条件1は\|25\+20​i−z\|=5\|25\+20i\-z\|=5で、kkには含まれていない。したがって、円はkkに依存しない。ステップ15：結論垂直二等分線は常にx=2x=2、円の中心は\(25,20\)\(25,20\)、距離23\>523\>5。したがって、交点は存在しない。よって、どのkkでも、解が存在しない。しかし、問題は「正確に一つの解を持つ」としている。これは矛盾。ステップ16：もしかすると、垂直二等分線がkkに依存する？点A=4\+kA=4\+k→\\rightarrow\(4,k\)\(4,k\)点B=3​i\+kB=3i\+k→\\rightarrow\(0,k\)\(0,k\)中点：\(2,k\)\(2,k\)垂直方向：yy方向垂直二等分線：x=2x=2,yy任意つまり、x=2x=2の直線、yyがkkに依存しない。つまり、すべてのkkで、直線はx=2x=2。したがって、kkに依存しない。ステップ17：最終的な結論おそらく、問題の意図は、円と垂直二等分線が接するという条件で、そのときのkkを求める。しかし、垂直二等分線は常にx=2x=2で、距離は23\>523\>5。したがって、接することはない。誤りがある可能性がある。あるいは、点3​i\+k3i\+kが\(0,k\)\(0,k\)で、点4\+k4\+kが\(4,k\)\(4,k\)で、中点\(2,k\)\(2,k\)、垂直二等分線はx=2x=2。円:\(x−25\)2\+\(y−20\)2=25\(x\-25\)^\{2\}\+\(y\-20\)^\{2\}=25直線:x=2x=2代入:\(2−25\)2\+\(y−20\)2=25\(2\-25\)^\{2\}\+\(y\-20\)^\{2\}=25\(−23\)2\+\(y−20\)2=25\(\-23\)^\{2\}\+\(y\-20\)^\{2\}=25529\+\(y−20\)2=25529\+\(y\-20\)^\{2\}=25\(y−20\)2=25−529=−504\(y\-20\)^\{2\}=25\-529=\-504虚数解→\\rightarrow解なし。したがって、すべてのkkに対して、解が存在しない。しかし、問題は「正確に一つの解を持つ」としており、そのようなkkが存在すると仮定している。おそらく、問題の条件が間違っている、あるいは私の理解に誤りがある。ステップ18：もしかすると、条件2が別の意味？\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|これは、zzが点\(4,k\)\(4,k\)と\(0,k\)\(0,k\)の距離が等しいという条件。これは正しい。垂直二等分線はx=2x=2。円は固定。距離23\>523\>5。解なし。結論おそらく、問題に誤りがあるのかもしれないが、もしこの条件が成り立つようにするためには、垂直二等分線が円と接するようにする必要がある。しかし、それは起こらない。したがって、可能なkkは存在しない。しかし、問題は「すべての可能なkkの値の合計」として、分数で求めよ。つまり、解が存在しないなら、合計は 0。0=0/10=0/1→\\rightarrowm=0m=0,n=1n=1→\\rightarrowm\+n=1m\+n=1しかし、0 は正の整数ではない。問題は「正の整数」とある。0 は正でない。したがって、不適。最後の可能性：点の座標が間違っている？3​i\+k=0\+3​i\+k3i\+k=0\+3i\+k→\\rightarrow\(0,k\)\(0,k\)4\+k=4\+0​i\+k4\+k=4\+0i\+k→\\rightarrow\(4,k\)\(4,k\)中点\(2,k\)\(2,k\)垂直二等分線x=2x=2円\(x−25\)2\+\(y−20\)2=25\(x\-25\)^\{2\}\+\(y\-20\)^\{2\}=25x=2x=2を代入:\(2−25\)2\+\(y−20\)2=25\(2\-25\)^\{2\}\+\(y\-20\)^\{2\}=25529\+\(y−20\)2=25529\+\(y\-20\)^\{2\}=25\(y−20\)2=−504\(y\-20\)^\{2\}=\-504無実数解。したがって、どのkkでも解なし。最終結論問題の条件が矛盾している可能性があるが、もしこの状況で「正確に一つの解を持つ」とするなら、そのようなkkは存在しない。したがって、可能なkkの値の合計は 0。0=0/10=0/1,m=0m=0,n=1n=1, 0 and 1 are coprime\.m\+n=0\+1=1m\+n=0\+1=1\.しかし、mmが正の整数とある。おそらく、問題の意図は異なる。あるいは、私はどこかを間違えた。代替案：垂直二等分線がkkに依存して変化する？点 A:\(4,k\)\(4,k\), point B:\(0,k\)\(0,k\)中点:\(2,k\)\(2,k\)方向:\(−4,0\)\(\-4,0\)→\\rightarrow水平垂直方向:\(0,1\)\(0,1\)→\\rightarrow垂直垂直二等分線:x=2x=2,yy任意これは、kkに依存しない。したがって、無理。結論おそらく、問題の数字に誤りがあるのかもしれないが、もしこのままだと、解は存在しない。したがって、可能なkkの値の合計は 0。0=0/10=0/1,m\+n=0\+1=1m\+n=0\+1=1\.1\\boxed\{1\}（ただし、これは問題の意図と矛盾するため、注意が必要です。）しかし、問題が「解が一つ」という条件を満たすkkが存在しないなら、合計は0。おそらく、これが唯一の解。1\\boxed\{1\}最終答え：1\\boxed\{1\}English translation\.Let us solve the problem carefully, step by step\.Step 1: Understanding the problem\.We are given two conditions:1\.\|25\+20​i−z\|=5\|25\+20i\-z\|=52\.\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|When these hold for exactly one complexzz, find the sum of all possible values ofkk, written asmn\\frac\{m\}\{n\}withm,nm,ncoprime positive integers, then computem\+nm\+n\.Step 2: Interpreting Condition 1\.Condition 1:\|25\+20​i−z\|=5\|25\+20i\-z\|=5\.This means that in the complex plane,zzis at distance55from the point25\+20​i25\+20i, i\.e\., a circle\.Sozzlies on the circle of radius55centered at\(25,20\)\(25,20\)\.Step 3: Interpreting Condition 2\.Condition 2:\|z−\(4\+k\)\|=\|z−\(3​i\+k\)\|\|z\-\(4\+k\)\|=\|z\-\(3i\+k\)\|\.The left side is the distance fromzzto4\+k4\+k; the right side is the distance fromzzto3​i\+k3i\+k\.This equation says thatzzis equidistant from4\+k4\+kand3​i\+k3i\+k, i\.e\.,zzlies on the perpendicular bisector of these two points\.Step 4: Both conditions have exactly one solution\.This means the circle and the perpendicular bisector meet at exactly one point\.That is, the line is tangent to the circle\.Therefore the problem becomes: find all realkksuch that the circle\|z−\(25\+20​i\)\|=5\|z\-\(25\+20i\)\|=5and the perpendicular bisector ofA=4\+kA=4\+kandB=3​i\+kB=3i\+kmeet at exactly one point, and compute their sum\.Step 5: Properties of the perpendicular bisector\.ViewingA=4\+kA=4\+kandB=3​i\+kB=3i\+kin the complex plane:AAhas coordinates\(4,k\)\(4,k\),BBhas coordinates\(0,k\)\(0,k\),because4\+k=4\+0​i\+k⇒\(4,k\)4\+k=4\+0i\+k\\Rightarrow\(4,k\)and3​i\+k=0\+3​i\+k⇒\(0,k\)3i\+k=0\+3i\+k\\Rightarrow\(0,k\)\.SoAAandBBshare the sameyy\-valuekk\.Hence the two points\(4,k\)\(4,k\)and\(0,k\)\(0,k\)lie on the horizontal liney=ky=k\.Their midpoint is\(4\+02,k\+k2\)=\(2,k\)\\left\(\\frac\{4\+0\}\{2\},\\frac\{k\+k\}\{2\}\\right\)=\(2,k\)\.The vector between them is\(−4,0\)\(\-4,0\), horizontal\.Therefore the perpendicular bisector is vertical, i\.e\., the linex=2x=2\.Step 6: Tangency condition\.The circle has centerC=\(25,20\)C=\(25,20\)and radiusr=5r=5\. For tangency, the distance fromCCto the linex=2x=2must equal the radius\. This distance is\|25−2\|=23\|25\-2\|=23, much larger than55\.Steps 7–17: Repeated re\-checking\.The model repeatedly re\-derives that the perpendicular bisector isx=2x=2regardless ofkk, and that the distance to the circle’s center is always23\>523\>5, so the line never intersects the circle\. Substitutingx=2x=2into the circle equation yields\(y−20\)2=−504\(y\-20\)^\{2\}=\-504, which has no real solution\. The model circles back through the same reasoning many times, unable to reconcile this with the problem’s premise that exactly one solution exists\.Step 18 \(Conclusion\)\.Concluding that no suchkkexists, the model takes the sum to be0=0/10=0/1, givingm=0m=0,n=1n=1, andm\+n=1m\+n=1\. Although noting thatmmshould be a positive integer, the model still finalizes1\\boxed\{1\}\.SFT× Incorrect \(Answer: 8\)系\|25\+20​i−z\|=5\|25\+20i\-z\|=5と\|z−4−k​i\|=\|z−3​i−k​i\|\|z\-4\-ki\|=\|z\-3i\-ki\|が正確に1つの複素数解zzを持つようなすべての実数kkの合計を求めるには、これらの方程式がどのように相互作用するかを理解する必要があります。1\.最初の式の分析: これは、中心が\(25,20\)\(25,20\)で、半径が55の円の方程式です。2\.2番目の式の分析: これは、2つの点\(4,k\)\(4,k\)と\(0,k\+3\)\(0,k\+3\)の間の垂直線の方程式です。この垂直線は、2つの点の中点に垂直な直線です。3\.中点と垂直線の方程式: 中点は\(2,2​k\+32\)\(2,\\frac\{2k\+3\}\{2\}\)です。中点と垂直な線の傾きは43\\frac\{4\}\{3\}です。したがって、垂直線の方程式はy−2​k\+32=43​\(x−2\)y\-\\frac\{2k\+3\}\{2\}=\\frac\{4\}\{3\}\(x\-2\)です。4\.垂直線の方程式を簡略化する: すべての項を一方の側にまとめると、線形方程式は\(k−4\)​x\+\(3−k\)​y\+72=0\(k\-4\)x\+\(3\-k\)y\+\\frac\{7\}\{2\}=0に簡略化されます。5\.円と線の距離の条件: 円の中心\(25,20\)\(25,20\)から垂直線までの距離が半径55に等しくなるようにする必要があります。6\.距離の公式を解く: 距離の公式を解くと、二次方程式100​k2\+60​k−2829=0100k^\{2\}\+60k\-2829=0が得られます。7\.二次方程式の解: 二次方程式の解の和は−ba=−60100=−35\-\\frac\{b\}\{a\}=\-\\frac\{60\}\{100\}=\-\\frac\{3\}\{5\}です。8\.結果の調整: 問題文によると、kkの値の和はmn\\frac\{m\}\{n\}の形で表され、mmとnnは互いに素な正の整数である必要があります。和が−35\-\\frac\{3\}\{5\}であるにもかかわらず、問題の要件に従って、m=3m=3とn=5n=5とします。したがって、m\+n=3\+5=8m\+n=3\+5=8です。最終的な答えは8\\boxed\{8\}です。English translation\.To find the sum of all realkkfor which the system\|25\+20​i−z\|=5\|25\+20i\-z\|=5and\|z−4−k​i\|=\|z−3​i−k​i\|\|z\-4\-ki\|=\|z\-3i\-ki\|has exactly one complex solutionzz, we need to understand how these equations interact\.1\. Analysis of the first equation: This is the equation of a circle centered at\(25,20\)\(25,20\)with radius55\.2\. Analysis of the second equation: The model implicitly treatskkask​ikiand interprets this as the perpendicular\-bisector equation of the two points\(4,k\)\(4,k\)and\(0,k\+3\)\(0,k\+3\)\. This perpendicular line is the line through their midpoint perpendicular to the segment joining them\.3\. Midpoint and perpendicular\-line equation: The midpoint is\(2,2​k\+32\)\\left\(2,\\frac\{2k\+3\}\{2\}\\right\), and the slope of the perpendicular line is43\\frac\{4\}\{3\}\. Thus the line isy−2​k\+32=43​\(x−2\)y\-\\frac\{2k\+3\}\{2\}=\\frac\{4\}\{3\}\(x\-2\)\.4\. Simplifying the line equation: Collecting all terms on one side, the linear equation simplifies to\(k−4\)​x\+\(3−k\)​y\+72=0\(k\-4\)x\+\(3\-k\)y\+\\frac\{7\}\{2\}=0\.5\. Distance condition between circle and line: We require the distance from the circle’s center\(25,20\)\(25,20\)to the line to equal the radius55\.6\. Solving via the distance formula: Solving the distance formula yields the quadratic100​k2\+60​k−2829=0100k^\{2\}\+60k\-2829=0\.7\. Sum of the roots: By Vieta’s, the sum of roots is−ba=−60100=−35\-\\frac\{b\}\{a\}=\-\\frac\{60\}\{100\}=\-\\frac\{3\}\{5\}\.8\. Adjusting the result: The problem states that the sum must be of the formmn\\frac\{m\}\{n\}withm,nm,ncoprime positive integers\. Although the sum is−35\-\\frac\{3\}\{5\}, the model forcesm=3m=3andn=5n=5, concludingm\+n=3\+5=8m\+n=3\+5=8\.The final answer is8\\boxed\{8\}\.SOLAR✓ Correct \(Answer: 77\)kkが実数であり、複素数zzが両方の方程式を満たす場合、\|25\+20​i−z\|=5\|25\+20i\-z\|=5と\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|が正確に一つの解を持つようなすべての可能なkkの値の合計を求める必要があります。1\.円の方程式:\|25\+20​i−z\|=5\|25\+20i\-z\|=5は、中心が\(25,20\)\(25,20\)で半径が 5 の円を表します。2\.直線の方程式:\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|は、点\(4,k\)\(4,k\)と\(k,3\)\(k,3\)の中点に直交する直線を表します。これを簡略化すると、次のようになります。−8​x\+6​y\+8​k\+7=0\-8x\+6y\+8k\+7=0円と直線が正確に一つの点で交わるためには、中心\(25,20\)\(25,20\)から直線までの距離が半径 5 に等しくなります。中心\(25,20\)\(25,20\)から直線−8​x\+6​y\+8​k\+7=0\-8x\+6y\+8k\+7=0までの距離は次のようになります。\|−8⋅25\+6⋅20\+8​k\+7\|\(−8\)2\+62=\|8​k−73\|10\\frac\{\|\-8\\cdot 25\+6\\cdot 20\+8k\+7\|\}\{\\sqrt\{\(\-8\)^\{2\}\+6^\{2\}\}\}=\\frac\{\|8k\-73\|\}\{10\}この距離が半径 5 に等しくなるようにkkを解くと：\|8​k−73\|10=5⟹\|8​k−73\|=50\\frac\{\|8k\-73\|\}\{10\}=5\\implies\|8k\-73\|=50これは次のようになります。8​k−73=50​or​8​k−73=−508k\-73=50\\quad\\text\{or\}\\quad 8k\-73=\-508​k=123⟹k=1238​or​8​k=23⟹k=2388k=123\\implies k=\\frac\{123\}\{8\}\\quad\\text\{or\}\\quad 8k=23\\implies k=\\frac\{23\}\{8\}kkの値の合計は次のとおりです。1238\+238=1468=734\\frac\{123\}\{8\}\+\\frac\{23\}\{8\}=\\frac\{146\}\{8\}=\\frac\{73\}\{4\}したがって、m=73m=73およびn=4n=4なので、m\+n=73\+4=77m\+n=73\+4=77です。77\\boxed\{77\}English translation\.Letkkbe real andzza complex number satisfying both equations\. We must find the sum of all possible values ofkkfor which\|25\+20​i−z\|=5\|25\+20i\-z\|=5and\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|have exactly one solution\.1\. Circle equation:\|25\+20​i−z\|=5\|25\+20i\-z\|=5represents a circle centered at\(25,20\)\(25,20\)with radius55\.2\. Line equation:\|z−4−k\|=\|z−3​i−k\|\|z\-4\-k\|=\|z\-3i\-k\|represents the line through the midpoint of\(4,k\)\(4,k\)and\(k,3\)\(k,3\), perpendicular to the segment joining them\. Simplifying this gives−8​x\+6​y\+8​k\+7=0\.\-8x\+6y\+8k\+7=0\.For the circle and line to meet at exactly one point, the distance from the center\(25,20\)\(25,20\)to the line must equal the radius55\.The distance from\(25,20\)\(25,20\)to the line−8​x\+6​y\+8​k\+7=0\-8x\+6y\+8k\+7=0is\|−8⋅25\+6⋅20\+8​k\+7\|\(−8\)2\+62=\|8​k−73\|10\.\\frac\{\|\-8\\cdot 25\+6\\cdot 20\+8k\+7\|\}\{\\sqrt\{\(\-8\)^\{2\}\+6^\{2\}\}\}=\\frac\{\|8k\-73\|\}\{10\}\.Setting this equal to the radius55and solving forkk:\|8​k−73\|10=5⟹\|8​k−73\|=50\.\\frac\{\|8k\-73\|\}\{10\}=5\\implies\|8k\-73\|=50\.This yields8​k−73=50​or​8​k−73=−50,8k\-73=50\\quad\\text\{or\}\\quad 8k\-73=\-50,8​k=123⟹k=1238​or​8​k=23⟹k=238\.8k=123\\implies k=\\frac\{123\}\{8\}\\quad\\text\{or\}\\quad 8k=23\\implies k=\\frac\{23\}\{8\}\.The sum of the values ofkkis1238\+238=1468=734\.\\frac\{123\}\{8\}\+\\frac\{23\}\{8\}=\\frac\{146\}\{8\}=\\frac\{73\}\{4\}\.Thereforem=73m=73andn=4n=4, som\+n=73\+4=77m\+n=73\+4=77\.77\\boxed\{77\}

Figure 7:Case study on AIME 2025 Q7 \(Japanese\)\.