Hidden Language Consistency Phenomena in Reasoning LLMs

arXiv cs.CL Papers
multilingual reasoning language-consistency quantization benchmark chain-of-thought

Summary

This paper studies multilingual reasoning models and reveals that language consistency in outputs can degrade or collapse with task difficulty, especially for less represented languages. It argues that evaluating multilingual capability requires jointly considering accuracy, language consistency, and task difficulty.

arXiv:2608.08447v1 Announce Type: new Abstract: Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended language while reasoning and responding. This omission conceals important multilingual behaviors that emerge as tasks become harder. In this paper, we study task difficulty, task accuracy, thinking-language consistency (TC), and answer-language consistency (AC) across reasoning models using PolyMath benchmark in eight languages and four difficulty levels. We uncover four findings: (1) language consistency exhibits four difficulty-dependent behaviors: output-language consistency remains aligned with input, remains misaligned, degrades gradually, or collapses abruptly. (2) We identify the language consistency breakdown effect, where increasing difficulty can cause a sudden drop in output-language consistency, especially in less strongly represented and non-Latin-script languages. (3) Due to this breakdown effect, accuracy can be preserved or even improved at a harder difficulty level as the model shifts to its internal dominant language. (4) Quantization can improve or degrade output-language consistency independently of its effect on accuracy, with GPTQ and AWQ often outperforming AutoRound under tolerance-based voting with {\epsilon} = 1.0. These results show that multilingual capability cannot be characterized by accuracy alone; reliable evaluation should jointly consider task accuracy, language consistency, and task difficulty for multilingual benchmarks.
Original Article
View Cached Full Text

Cached at: 08/11/26, 08:08 AM 

# Hidden Language Consistency Phenomena in Reasoning LLMs
Source: [https://arxiv.org/abs/2608.08447](https://arxiv.org/abs/2608.08447)
[View PDF](https://arxiv.org/pdf/2608.08447)

> Abstract:Multilingual reasoning models are commonly evaluated by whether they arrive at the correct answer, but not by whether they preserve the intended language while reasoning and responding\. This omission conceals important multilingual behaviors that emerge as tasks become harder\. In this paper, we study task difficulty, task accuracy, thinking\-language consistency \(TC\), and answer\-language consistency \(AC\) across reasoning models using PolyMath benchmark in eight languages and four difficulty levels\. We uncover four findings: \(1\) language consistency exhibits four difficulty\-dependent behaviors: output\-language consistency remains aligned with input, remains misaligned, degrades gradually, or collapses abruptly\. \(2\) We identify the language consistency breakdown effect, where increasing difficulty can cause a sudden drop in output\-language consistency, especially in less strongly represented and non\-Latin\-script languages\. \(3\) Due to this breakdown effect, accuracy can be preserved or even improved at a harder difficulty level as the model shifts to its internal dominant language\. \(4\) Quantization can improve or degrade output\-language consistency independently of its effect on accuracy, with GPTQ and AWQ often outperforming AutoRound under tolerance\-based voting with \{\\epsilon\} = 1\.0\. These results show that multilingual capability cannot be characterized by accuracy alone; reliable evaluation should jointly consider task accuracy, language consistency, and task difficulty for multilingual benchmarks\.

## Submission history

From: Muhammad Ali Shafique \[[view email](https://arxiv.org/show-email/df0ebd98/2608.08447)\] **\[v1\]**Sun, 9 Aug 2026 03:30:14 UTC \(3,709 KB\)

Similar Articles

Large Reasoning Models Are (Not Yet) Multilingual Latent Reasoners

arXiv cs.CL

This paper investigates multilingual latent reasoning in large reasoning models across 11 languages, revealing that while latent reasoning capabilities exist, they are unevenly distributed—stronger in resource-rich languages and weaker in low-resource ones. The study finds that despite surface-level differences, the internal reasoning mechanisms are largely aligned with an English-centered pathway.

Rethinking the Multilingual Reasoning Gap with Layer Swap

arXiv cs.CL

This paper revisits the multilingual reasoning gap in LLMs, finding it smaller than previously reported under comparable supervision. It introduces Layer Swap, which transfers mid-layer weights from an English reasoning specialist to native language specialists, nearly closing the gap while preserving native-language chain-of-thought.

mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?

arXiv cs.CL

Introduces mmPISA-bench, a compact multilingual reasoning benchmark derived from PISA, and evaluates proprietary LLMs across 43 languages, finding that they reason effectively with some performance variations, and that machine-translated questions do not degrade accuracy.