LLMs Get Smarter from Targeted Synthetic Multilingual Data
Summary
HOTFIXR is a data generation framework that targets multilingual reasoning weaknesses in LLMs to improve cross-lingual performance using synthetic data without sacrificing overall capability.
View Cached Full Text
Cached at: 08/20/26, 08:06 PM
Paper page - LLMs Get Smarter from Targeted Synthetic Multilingual Data
Source: https://huggingface.co/papers/2608.15964
Abstract
HOTFIXR is a data generation framework that targets multilingual reasoning weaknesses to improve cross-lingual performance without sacrificing overall capability.
Language-specific competency(LSC) is the phenomenon of a language model performing better or worse depending on the language of the prompt. In other words, a language model outputs different (and potentially incorrect) responses to the same semantic query when prompted in different languages. Prior work attributes this to an internal misalignment of semantic representation across languages. Currently, there are two main approaches to address LSC in the literature: (1) routing all queries through English, improving performance, but limiting language expressivity to English; or (2) training on language-balanced data, equalizing model performance across languages, but reducing overall performance. In this work, we take a data centric perspective and introduce HOTFIXR: Hardness Optimized Training data For Improving X-Lingual Reasoning. It is a data generation framework that uses models to probe and learn a student model’s multilingual weaknesses, and generates data to mitigate them. HOTFIXR can generatemultilingual synthetic training datathat can improve multilingual performance. We evaluate on three in-distribution tasks, threeout-of-distributiontasks, and fourout-of-distributionlanguages. On average, HOTFIXR (1) improves in-distribution performance by 6.2%, (2) reducescatastrophic forgetting(induced by fine-tuning) on OOD tasks by 3.7%, and (3) on OOD languages by 7.1%. Overall, as many real-world applications requires multilingual LLMs, our work contributes to the efforts of making LLMs multilingually proficient. We will release code upon acceptance.
View arXiv pageView PDFAdd to collection
Get this paper in your agent:
hf papers read 2608\.15964
Don’t have the latest CLI?curl \-LsSf https://hf\.co/cli/install\.sh \| bash
Models citing this paper0
No model linking this paper
Cite arxiv.org/abs/2608.15964 in a model README.md to link it from this page.
Datasets citing this paper0
No dataset linking this paper
Cite arxiv.org/abs/2608.15964 in a dataset README.md to link it from this page.
Spaces citing this paper0
No Space linking this paper
Cite arxiv.org/abs/2608.15964 in a Space README.md to link it from this page.
Collections including this paper0
No Collection including this paper
Add this paper to acollectionto link it from this page.
Similar Articles
LLMs Improving LLMs: Agentic Discovery for Test-Time Scaling
This paper introduces AutoTTS, an environment-driven framework that automates the discovery of test-time scaling strategies for LLMs by formulating it as controller synthesis. It demonstrates improved accuracy-cost tradeoffs on mathematical reasoning benchmarks with minimal computational overhead.
Rethinking the Multilingual Reasoning Gap with Layer Swap
This paper revisits the multilingual reasoning gap in LLMs, finding it smaller than previously reported under comparable supervision. It introduces Layer Swap, which transfers mid-layer weights from an English reasoning specialist to native language specialists, nearly closing the gap while preserving native-language chain-of-thought.
Multilingual and Multimodal LLMs in the Wild: Building for Low-Resource Languages
This tutorial paper provides an overview of building multilingual and multimodal LLMs for low-resource languages, covering data creation, model alignment, fine-tuning, and evaluation, with a focus on practical recipes and hands-on resources.
Synthetic Consumer Insight Generation with Large Language Models
This research examines whether LLMs can generate synthetic consumer data for projective techniques, comparing human and LLM responses on city tourism perceptions and finding substantial overlap but differences in style and diversity.
mmPISA-bench: Do LLMs Reason Equally Well Across 43 Languages?
Introduces mmPISA-bench, a compact multilingual reasoning benchmark derived from PISA, and evaluates proprietary LLMs across 43 languages, finding that they reason effectively with some performance variations, and that machine-translated questions do not degrade accuracy.