PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Summary
PluraMath extends the PolyMath dataset to 18 underrepresented languages, providing a human-validated benchmark for evaluating multilingual mathematical reasoning in LLMs. The paper reveals a persistent performance gap between high-resource and low-resource languages across 27 models.
View Cached Full Text
Cached at: 07/08/26, 04:42 AM
# PluraMath: Extending Mathematical Reasoning Evaluation Beyond High-Resource Languages
Source: [https://arxiv.org/abs/2607.05992](https://arxiv.org/abs/2607.05992)
Authors:[Daryna Dementieva](https://arxiv.org/search/cs?searchtype=author&query=Dementieva,+D),[Nikolay Babakov](https://arxiv.org/search/cs?searchtype=author&query=Babakov,+N),[Kathy Hämmerl](https://arxiv.org/search/cs?searchtype=author&query=H%C3%A4mmerl,+K),[Ilseyar Alimova](https://arxiv.org/search/cs?searchtype=author&query=Alimova,+I),[Jindřich Libovický](https://arxiv.org/search/cs?searchtype=author&query=Libovick%C3%BD,+J),[Shu Okabe](https://arxiv.org/search/cs?searchtype=author&query=Okabe,+S),[Miras Baisbay](https://arxiv.org/search/cs?searchtype=author&query=Baisbay,+M),[Lukas Edman](https://arxiv.org/search/cs?searchtype=author&query=Edman,+L),[Abrorkhon Inomkhujaev](https://arxiv.org/search/cs?searchtype=author&query=Inomkhujaev,+A),[Antonia Karamolegkou](https://arxiv.org/search/cs?searchtype=author&query=Karamolegkou,+A),[Mateusz Lango](https://arxiv.org/search/cs?searchtype=author&query=Lango,+M),[Volkan Özer](https://arxiv.org/search/cs?searchtype=author&query=%C3%96zer,+V),[Nikola Selic](https://arxiv.org/search/cs?searchtype=author&query=Selic,+N),[Subhankar Swain](https://arxiv.org/search/cs?searchtype=author&query=Swain,+S),[Tsedeniya Kinfe Temesgen](https://arxiv.org/search/cs?searchtype=author&query=Temesgen,+T+K),[Galit Bary Weisberg](https://arxiv.org/search/cs?searchtype=author&query=Weisberg,+G+B),[Alexander Fraser](https://arxiv.org/search/cs?searchtype=author&query=Fraser,+A)
[View PDF](https://arxiv.org/pdf/2607.05992)
> Abstract:Mathematical reasoning has become a central task for evaluating and tuning reasoning Large Language Models \(LLMs\), yet existing benchmarks remain heavily biased toward high\-resource languages, with English and Chinese dominating both pre\-training corpora and evaluation suites\. The recently released PolyMath \(Wang et al\., 2025\) dataset represents a significant step forward, yet its coverage is still limited to 18 only high\-resource languages\. To address this gap, we introduce PluraMath, an extension of PolyMath to 18 additional \{underrepresented languages spanning 6 language families \-\- ranging from mid\-resource to extreme low\-resource settings\. We constructed the dataset through a human\-curated pipeline, where native speakers thoroughly validated pre\-computed translations\. Using PluraMath, we then benchmark 27 reasoning LLMs across four model scales \-\- small, mid\-size, large, and closed\-source ensembles \-\- probing the multilingual mathematical reasoning capabilities of state\-of\-the\-art models under diverse linguistic conditions\. Our fine\-grained analysis confirms a persistent gap in mathematical reasoning performance between high\-resource and underrepresented languages, with stronger results largely associated with better instruction\-following ability\. We fully open\-source our dataset, data acquisition pipeline, and evaluation framework, with the goal of lowering the barrier to multilingual benchmark development for underrepresented communities\.
## Submission history
From: Daryna Dementieva \[[view email](https://arxiv.org/show-email/4966295c/2607.05992)\] **\[v1\]**Tue, 7 Jul 2026 08:25:29 UTC \(6,751 KB\)Similar Articles
mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?
Introduces mmPISA-bench, a compact multilingual reasoning benchmark derived from PISA, and evaluates proprietary LLMs across 43 languages, finding that they reason effectively with some performance variations, and that machine-translated questions do not degrade accuracy.
Large Language Models for Math Education in Low-Resource Languages: A Study in Sinhala and Tamil
This paper evaluates the mathematical reasoning capabilities of large language models in Sinhala and Tamil, two low-resource South Asian languages, using a parallel dataset of independently authored problems. The study demonstrates that while basic arithmetic transfers well across languages, complex reasoning tasks show significant performance degradation in non-English languages, with implications for deploying AI tutoring tools in multilingual educational contexts.
LLM Parameters for Math Across Languages: Shared or Separate?
This paper presents a cross-lingual mechanistic analysis of mathematical reasoning in LLMs, finding partial overlap of math-associated parameters across languages, concentrated in intermediate layers. English has the largest set of math-relevant parameters, while lower-resource languages have smaller sets.
MathNet: a Global Multimodal Benchmark for Mathematical Reasoning and Retrieval
MathNet is a large-scale multilingual multimodal benchmark of 30,676 Olympiad-level math problems spanning 47 countries and 17 languages, designed to evaluate mathematical reasoning and retrieval in generative and embedding-based models. Even state-of-the-art models like Gemini and GPT-5 struggle with the benchmark, highlighting significant room for improvement in mathematical AI.
Mathematical Reasoning in Large Language Models: Benchmarks, Architectures, Evaluation, and Open Challenges
This survey synthesizes recent advancements in mathematical reasoning with large language models, covering benchmarks, architectures, training strategies, and evaluation protocols. It identifies key challenges such as reasoning faithfulness and benchmark biases.