Hidden Language Consistency Phenomena in Reasoning LLMs

arXiv cs.CL Papers
multilingual reasoning language-consistency quantization benchmark chain-of-thought

Summary

This paper studies multilingual reasoning models and reveals that language consistency in outputs can degrade or collapse with task difficulty, especially for less represented languages. It argues that evaluating multilingual capability requires jointly considering accuracy, language consistency, and task difficulty.

arXiv:2608.08447v1 Announce Type: new Abstract: Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended language while reasoning and responding. This omission conceals important multilingual behaviors that emerge as tasks become harder. In this paper, we study task difficulty, task accuracy, thinking-language consistency (TC), and answer-language consistency (AC) across reasoning models using PolyMath benchmark in eight languages and four difficulty levels. We uncover four findings: (1) language consistency exhibits four difficulty-dependent behaviors: output-language consistency remains aligned with input, remains misaligned, degrades gradually, or collapses abruptly. (2) We identify the language consistency breakdown effect, where increasing difficulty can cause a sudden drop in output-language consistency, especially in less strongly represented and non-Latin-script languages. (3) Due to this breakdown effect, accuracy can be preserved or even improved at a harder difficulty level as the model shifts to its internal dominant language. (4) Quantization can improve or degrade output-language consistency independently of its effect on accuracy, with GPTQ and AWQ often outperforming AutoRound under tolerance-based voting with {\epsilon} = 1.0. These results show that multilingual capability cannot be characterized by accuracy alone; reliable evaluation should jointly consider task accuracy, language consistency, and task difficulty for multilingual benchmarks.
Original Article
View Cached Full Text

Cached at: 08/11/26, 08:08 AM 

# Hidden Language Consistency Phenomena in Reasoning LLMs
Source: [https://arxiv.org/abs/2608.08447](https://arxiv.org/abs/2608.08447)
[View PDF](https://arxiv.org/pdf/2608.08447)

> Abstract:Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended language while reasoning and responding\. This omission conceals important multilingual behaviors that emerge as tasks become harder\. In this paper, we study task difficulty, task accuracy, thinking\-language consistency \(TC\), and answer\-language consistency \(AC\) across reasoning models using PolyMath benchmark in eight languages and four difficulty levels\. We uncover four findings: \(1\) language consistency exhibits four difficulty\-dependent behaviors: output\-language consistency remains aligned with input, remains misaligned, degrades gradually, or collapses abruptly\. \(2\) We identify the language consistency breakdown effect, where increasing difficulty can cause a sudden drop in output\-language consistency, especially in less strongly represented and non\-Latin\-script languages\. \(3\) Due to this breakdown effect, accuracy can be preserved or even improved at a harder difficulty level as the model shifts to its internal dominant language\. \(4\) Quantization can improve or degrade output\-language consistency independently of its effect on accuracy, with GPTQ and AWQ often outperforming AutoRound under tolerance\-based voting with \{\\epsilon\} = 1\.0\. These results show that multilingual capability cannot be characterized by accuracy alone; reliable evaluation should jointly consider task accuracy, language consistency, and task difficulty for multilingual benchmarks\.

## Submission history

From: Muhammad Ali Shafique \[[view email](https://arxiv.org/show-email/df0ebd98/2608.08447)\] **\[v1\]**Sun, 9 Aug 2026 03:30:14 UTC \(3,709 KB\)

Similar Articles

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners

arXiv cs.CL

This paper investigates multilingual latent reasoning in large reasoning models across 11 languages, revealing that while latent reasoning capabilities exist, they are unevenly distributed—stronger in resource-rich languages and weaker in low-resource ones. The study finds that despite surface-level differences, the internal reasoning mechanisms are largely aligned with an English-centered pathway.

Skill Issue: Are Skills Language-Invariant in LLMs?

Hugging Face Daily Papers

This paper quantifies cross-lingual skill inconsistencies in large language models through multilingual self-play in text-based games, revealing significant variations in performance across languages that can be partially mitigated by altering intermediate reasoning language.

Rethinking the Multilingual Reasoning Gap with Layer Swap

arXiv cs.CL

This paper revisits the multilingual reasoning gap in LLMs, finding it smaller than previously reported under comparable supervision. It introduces Layer Swap, which transfers mid-layer weights from an English reasoning specialist to native language specialists, nearly closing the gap while preserving native-language chain-of-thought.

Investigating the Role of Reasoning-Language Alignment in Monolingual Retrieval-Augmented Generation

arXiv cs.CL

This paper investigates whether forcing LLMs to reason in the same language as the query and retrieved documents improves monolingual non-English RAG performance. Using a German RAG testbed built over the fictional world of The Dark Eye, the authors find that aligning the reasoning language with the evidence helps (German reasoning beats French despite the model's stronger French proficiency), but native unconstrained English reasoning still matches the aligned setting, highlighting the need for truly multilingual reasoning models.