Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations

arXiv cs.CL Papers
multilingual-llm robustness structural-perturbations mathematical-reasoning indian-languages benchmark-dataset

Summary

This paper investigates the structural sensitivity of multilingual large language models to semantics-preserving perturbations in Hindi and Malayalam, showing significant degradation in mathematical reasoning performance and introducing the IndicReStruct benchmark for evaluation.

arXiv:2609.03511v1 Announce Type: new Abstract: Large Language Models (LLMs) demonstrate strong multilingual reasoning performance, yet their robustness to semantics-preserving structural variation remains underexplored, particularly for relatively free word-order languages. We investigate the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam: constrained constituent reordering and active-passive voice transformation. We introduce a benchmark dataset IndicReStruct, with two variants, GSM8K-Reordered and GSM8K-Voice, constructed from GSM8K while preserving semantic meaning. Across six state-of-the-art LLMs and multiple prompting strategies, we observe consistent and significant degradation in mathematical reasoning performance under structurally perturbed inputs. To further understand these failures, we perform qualitative error analysis and mechanistic interpretability experiments using residual-stream activation patching. Our analyses show that reasoning failures frequently arise from disruptions in entity-quantity alignment and that intermediate transformer layers contribute most strongly toward reasoning restoration. Overall, our findings suggest that current multilingual LLMs remain highly sensitive to surface syntactic realization and lack robust compositional invariance under structurally different but semantically equivalent inputs.
Original Article
View Cached Full Text

Cached at: 09/04/26, 06:01 AM 

# Lost in Reordering: Structural Sensitivity of Multilingual LLMs under Semantics-Preserving Perturbations
Source: [https://arxiv.org/abs/2609.03511](https://arxiv.org/abs/2609.03511)
[View PDF](https://arxiv.org/pdf/2609.03511)

> Abstract:Large Language Models \(LLMs\) demonstrate strong multilingual reasoning performance, yet their robustness to semantics\-preserving structural variation remains underexplored, particularly for relatively free word\-order languages\. We investigate the structural sensitivity of multilingual LLMs using two linguistically grounded perturbation settings in Hindi and Malayalam: constrained constituent reordering and active\-passive voice transformation\. We introduce a benchmark dataset IndicReStruct, with two variants, GSM8K\-Reordered and GSM8K\-Voice, constructed from GSM8K while preserving semantic meaning\. Across six state\-of\-the\-art LLMs and multiple prompting strategies, we observe consistent and significant degradation in mathematical reasoning performance under structurally perturbed inputs\. To further understand these failures, we perform qualitative error analysis and mechanistic interpretability experiments using residual\-stream activation patching\. Our analyses show that reasoning failures frequently arise from disruptions in entity\-quantity alignment and that intermediate transformer layers contribute most strongly toward reasoning restoration\. Overall, our findings suggest that current multilingual LLMs remain highly sensitive to surface syntactic realization and lack robust compositional invariance under structurally different but semantically equivalent inputs\.

## Submission history

From: N J Karthika \[[view email](https://arxiv.org/show-email/3dcace1c/2609.03511)\] **\[v1\]**Thu, 3 Sep 2026 08:10:43 UTC \(1,839 KB\)

Similar Articles

Hidden Language Consistency Phenomena in Reasoning LLMs

arXiv cs.CL

This paper studies multilingual reasoning models and reveals that language consistency in outputs can degrade or collapse with task difficulty, especially for less represented languages. It argues that evaluating multilingual capability requires jointly considering accuracy, language consistency, and task difficulty.

Safety Alignment Illusion: The Cross-Lingual Safety Gap in LLMs

arXiv cs.AI

The paper introduces INCLUDE, a multilingual evaluation benchmark to quantify Indian-centric socio-cultural biases in LLMs, revealing that non-English Indian languages exhibit higher bias than English, indicating cross-lingual safety alignment gaps.

Fragile Thoughts: How Large Language Models Handle Chain-of-Thought Perturbations

arXiv cs.CL

This paper presents a comprehensive empirical evaluation of how large language models handle corruptions in chain-of-thought reasoning steps, testing 13 models across 5 perturbation types (MathError, UnitConversion, Sycophancy, SkippedSteps, ExtraSteps) on mathematical reasoning tasks. The findings reveal heterogeneous vulnerability patterns with implications for deploying LLMs in multi-stage reasoning pipelines.