Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization

arXiv cs.CL Papers
multilingual-ai counterfactual-explanations preference-optimization interpretability dpo llm-alignment

Summary

The paper introduces Macro, a preference alignment framework using DPO to improve the validity and minimality of self-generated counterfactual explanations across multiple languages.

arXiv:2605.11632v1 Announce Type: new Abstract: Self-generated counterfactual explanations (SCEs) are minimally modified inputs (minimality) generated by large language models (LLMs) that flip their own predictions (validity), offering a causally grounded approach to unraveling black-box LLM behavior. Yet extending them beyond English remains challenging: existing methods struggle to produce valid SCEs in non-dominant languages, and a persistent trade-off between validity and minimality undermines explanation quality. We introduce Macro, a preference alignment framework that applies Direct Preference Optimization (DPO) to multilingual SCE generation, using a composite scoring function to construct preference pairs that effectively translate the trade-off into measurable preference signals. Experiments across four LLMs and seven typologically diverse languages show that Macro improves validity by 12.55\% on average over the chain-of-thought baseline without degrading minimality, while avoiding the severe minimality violations of the translation-based baseline. Compared to supervised fine-tuning, Macro achieves superior performance on both metrics, confirming that explicit preference optimization is essential for balancing this trade-off. Further analyses reveal that Macro increases cross-lingual perturbation alignment and mitigates common generation errors. Our results highlight preference optimization as a promising direction for enhancing multilingual model explanations.
Original Article
View Cached Full Text

Cached at: 05/13/26, 06:16 AM 

# Enhancing Multilingual Counterfactual Generation through Alignment-as-Preference Optimization
Source: [https://arxiv.org/abs/2605.11632](https://arxiv.org/abs/2605.11632)
[View PDF](https://arxiv.org/pdf/2605.11632)

> Abstract:Self\-generated counterfactual explanations \(SCEs\) are minimally modified inputs \(minimality\) generated by large language models \(LLMs\) that flip their own predictions \(validity\), offering a causally grounded approach to unraveling black\-box LLM behavior\. Yet extending them beyond English remains challenging: existing methods struggle to produce valid SCEs in non\-dominant languages, and a persistent trade\-off between validity and minimality undermines explanation quality\. We introduce Macro, a preference alignment framework that applies Direct Preference Optimization \(DPO\) to multilingual SCE generation, using a composite scoring function to construct preference pairs that effectively translate the trade\-off into measurable preference signals\. Experiments across four LLMs and seven typologically diverse languages show that Macro improves validity by 12\.55\\% on average over the chain\-of\-thought baseline without degrading minimality, while avoiding the severe minimality violations of the translation\-based baseline\. Compared to supervised fine\-tuning, Macro achieves superior performance on both metrics, confirming that explicit preference optimization is essential for balancing this trade\-off\. Further analyses reveal that Macro increases cross\-lingual perturbation alignment and mitigates common generation errors\. Our results highlight preference optimization as a promising direction for enhancing multilingual model explanations\.

## Submission history

From: Yilong Wang \[[view email](https://arxiv.org/show-email/34bf1a69/2605.11632)\] **\[v1\]**Tue, 12 May 2026 06:56:18 UTC \(9,700 KB\)

Similar Articles

Mechanistic Analysis of Alignment Algorithms in Language Models

arXiv cs.LG

This paper presents a systematic mechanistic analysis of six preference optimization methods (PPO, DPO, SimPO, ORPO, GRPO, KTO) across three open-weight model families, using probing and sparse autoencoders to reveal how alignment algorithms reshape internal representations in qualitatively distinct ways.

Direct Preference Optimization for English-Mandarin Code-Switching Speech Recognition in Audio LLMs

arXiv cs.CL

This paper applies Direct Preference Optimization (DPO) to align Audio LLMs for transcribing English-Mandarin code-switching speech, achieving up to 89.6% MER reduction in-distribution and 20% out-of-distribution. It identifies three failure modes—language omission, translation instead of transcription, and hallucination—and shows that preference-based alignment effectively elicits correct code-switching behavior from multilingual Audio LLMs.

CroCo: Cross-Lingual Contrastive Preference Tuning on Self-Generations

arXiv cs.CL

This paper introduces CroCo, a method for cross-lingual contrastive preference tuning on self-generated responses, showing that a reward model trained on English preferences can effectively rank responses in other languages, improving model performance across 14 languages without language-specific annotations.

PolyAlign: Conditional Human-Distribution Alignment

arXiv cs.CL

PolyAlign is a distribution-aware alignment framework that aligns language models to context-specific human response distributions rather than a single global style, improving naturalness and faithfulness across bilingual settings.