Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation

# Reference-Free Post-Training of Open Large Language Models for Multilingual Machine Translation
Source: [https://arxiv.org/html/2608.10812](https://arxiv.org/html/2608.10812)
Chris Han,Pengzhi Gao,Pei Fu,Jian Luan Xiaomi Inc\., Beijing, China

###### Abstract

We study reference\-free post\-training for multilingual machine translation with open large language models\. Starting from the supervised\-finetuned MiLMMT\-46\-v0\.1 modelsShanget al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib5)\), we apply Group Relative Policy Optimization \(GRPO\)Shaoet al\.\([2024](https://arxiv.org/html/2608.10812#bib.bib4)\)with a reward that averages two reference\-free quality estimation models and is gated by language identification\. We then linearly interpolate the supervised fine\-tuning \(SFT\) and reinforcement learning \(RL\) model checkpoints to obtain MiLMMT\-46\-v1\.0\. Across 46 languages, the resulting models consistently improve translation quality over their SFT counterparts, outperform strong recent open baselines including Seed\-XChenget al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib14)\), HY\-MT2Zhenget al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib17)\), and TranslateGemmaFinkelsteinet al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib18)\), and achieve leading reference\-free scores against evaluated proprietary systems such as Google Translate, Gemini 3 Pro, and GPT\-5\. We further investigate on\-policy distillation and find that it reaches, but does not surpass, the quality frontier achieved by RL with checkpoint interpolation\. We release the models and code to facilitate future research\.111Models are released at[https://huggingface\.co/collections/xiaomi\-research/milmmt\-46](https://huggingface.co/collections/xiaomi-research/milmmt-46)\. Codes are released at[https://github\.com/xiaomi\-research/gemmax](https://github.com/xiaomi-research/gemmax)\.

Reference\-Free Post\-Training of Open Large Language Models for Multilingual Machine Translation

Chris Han, Pengzhi Gao††thanks:Corresponding author:gaopengzhi@xiaomi\.com\., Pei Fu, Jian LuanXiaomi Inc\., Beijing, China

## 1Introduction

Open large language models \(LLMs\), including the Gemma and Qwen familiesGemma Team \([2026](https://arxiv.org/html/2608.10812#bib.bib2)\); Qwen Team \([2025](https://arxiv.org/html/2608.10812#bib.bib3)\), have recently become strong foundations for multilingual machine translation \(MT\)\. Through continual pretraining and supervised finetuning \(SFT\), recent work has adapted these models into capable multilingual translation systems\. In particular, the MiLMMT\-46\-v0\.1 modelsShanget al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib5)\)achieve strong performance across4646languages, providing a strong open starting point for further post\-training\.

Despite these advances, further improving multilingual MT with supervised learning remains constrained by the scarcity and uneven coverage of high\-quality parallel data, particularly for low\-resource languages and non\-English\-centric translation directions\. By contrast, source\-side text is far more abundant and easier to collect across languages\. This gap motivates reference\-free post\-training methods that can exploit source\-side data without aligned target sentences\.

Reinforcement learning \(RL\) provides a natural framework for this setting because it can optimize sequence\-level rewards on model\-generated translations rather than token\-level likelihood against references\. Group Relative Policy Optimization \(GRPO\)Shaoet al\.\([2024](https://arxiv.org/html/2608.10812#bib.bib4)\)has recently been applied to multilingual translation with carefully designed rewards, as exemplified by Tower\+Reiet al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib12)\)and HY\-MT2Zhenget al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib17)\)\. Reference\-free quality estimation \(QE\) models can score a source sentence and its candidate translation without a reference, but imperfect estimators may encourage reward hackingLiuet al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib20)\)\. A language\-identification gate can suppress wrong\-language outputs, while SFT–RL checkpoint interpolation can mitigate reward\-induced drift and preserve behavior learned during SFT\. These considerations motivate our combination of reference\-free RL with checkpoint interpolation\.

Building on the MiLMMT\-46\-v0\.1 models, we study this approach across three model scales and4646languages\. Our experiments show that reference\-free RL consistently improves learned quality metrics over SFT and compares favorably with strong recent open and proprietary systems, while checkpoint interpolation provides a controllable trade\-off between reference\-free quality and reference\-based performance\. We further investigate on\-policy distillation \(OPD\)Agarwalet al\.\([2024](https://arxiv.org/html/2608.10812#bib.bib27)\); Lu and Lab \([2025](https://arxiv.org/html/2608.10812#bib.bib26)\)as an alternative post\-training approach and find that it reaches, but does not surpass, the quality frontier achieved by RL with checkpoint interpolation\. We release the resulting MiLMMT\-46\-v1\.0 models and code to facilitate future research\.

## 2Methodology

Starting from the supervised\-finetuned MiLMMT\-46\-v0\.1 models, we use Group Relative Policy Optimization \(GRPO\) to optimize a reference\-free reward based on two quality estimation models and a language\-identification gate\. We then linearly interpolate the SFT and RL checkpoints to retain the behavior learned during supervised finetuning while incorporating the improvements from RL\.

#### Training objective\.

We adopt GRPO for the RL stage\. For each source sentencexx, we sample a group ofGGcandidate translations\{y1,…,yG\}\\\{y\_\{1\},\\dots,y\_\{G\}\\\}from the old policyπθold\\pi\_\{\\theta\_\{\\text\{old\}\}\}\. LetRiR\_\{i\}denote the reward assigned to candidateyiy\_\{i\}, and letπref\\pi\_\{\\text\{ref\}\}denote the fixed SFT reference policy\. We maximize the following objective:

𝒥​\(θ\)=\\displaystyle\\mathcal\{J\}\(\\theta\)=𝔼x,\{yi\}\[1G∑i=1G1\|yi\|∑t=1\|yi\|\{min\[ri,tA^i,t,\\displaystyle\\mathbb\{E\}\_\{x,\\,\\\{y\_\{i\}\\\}\}\\bigg\[\\tfrac\{1\}\{G\}\\textstyle\\sum\_\{i=1\}^\{G\}\\tfrac\{1\}\{\|y\_\{i\}\|\}\\sum\_\{t=1\}^\{\|y\_\{i\}\|\}\\Big\\\{\\min\\big\[r\_\{i,t\}\\,\\hat\{A\}\_\{i,t\},clip\(ri,t,1−ϵ,1\+ϵ\)A^i,t\]−β𝔻KL\(πθ∥πref\)\}\],\\displaystyle\\operatorname\{clip\}\(r\_\{i,t\},1\{\-\}\\epsilon,1\{\+\}\\epsilon\)\\,\\hat\{A\}\_\{i,t\}\\big\]\-\\beta\\,\\mathbb\{D\}\_\{\\text\{KL\}\}\(\\pi\_\{\\theta\}\\\|\\pi\_\{\\text\{ref\}\}\)\\Big\\\}\\bigg\],where

ri,t​\(θ\)=πθ​\(yi,t∣x,yi,<t\)πθold​\(yi,t∣x,yi,<t\)r\_\{i,t\}\(\\theta\)=\\frac\{\\pi\_\{\\theta\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}\{\\pi\_\{\\theta\_\{\\mathrm\{old\}\}\}\(y\_\{i,t\}\\mid x,y\_\{i,<t\}\)\}is the token\-level importance ratio,A^i,t\\hat\{A\}\_\{i,t\}is the group\-normalized advantage computed from the rewards\{Ri\}i=1G\\\{R\_\{i\}\\\}\_\{i=1\}^\{G\}of the sampled translations,ϵ\\epsiloncontrols the clipping range, andβ\\betacontrols the Kullback–Leibler \(KL\) penalty that regularizes the RL policy toward the SFT reference policy\.

#### Reward design\.

Given a source sentencexx, a candidate translationyy, and the intended target languageℓ\\ell, we predict the language ofyy, denoted byℓ^​\(y\)\\hat\{\\ell\}\(y\), using OpenLID\-v3Fedorovaet al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib22)\)\. LetsXs\_\{\\mathrm\{X\}\}andsKs\_\{\\mathrm\{K\}\}denote the scores assigned to\(x,y\)\(x,y\)by XCOMET222[https://huggingface\.co/Unbabel/XCOMET\-XXL](https://huggingface.co/Unbabel/XCOMET-XXL)Guerreiroet al\.\([2024](https://arxiv.org/html/2608.10812#bib.bib8)\)and COMETKiwi333[https://huggingface\.co/Unbabel/wmt23\-cometkiwi\-da\-xxl](https://huggingface.co/Unbabel/wmt23-cometkiwi-da-xxl)Reiet al\.\([2023](https://arxiv.org/html/2608.10812#bib.bib9)\), respectively\. We define the reward as

R​\(x,y,ℓ\)=\{12​\(sX\+sK\),ℓ^​\(y\)=ℓ,0,otherwise\.R\(x,y,\\ell\)=\\begin\{cases\}\\tfrac\{1\}\{2\}\\big\(s\_\{\\mathrm\{X\}\}\+s\_\{\\mathrm\{K\}\}\\big\),&\\hat\{\\ell\}\(y\)=\\ell,\\\\ 0,&\\text\{otherwise\}\.\\end\{cases\}This language\-identification gate prevents fluent but wrong\-language outputs from receiving high QE rewards, mitigating a known form of reward hackingLiuet al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib20)\)\.

#### RL data\.

We derive the RL dataset from the MiLMMT SFT data by retaining each example’s source sentence and translation direction while discarding its reference translation\. This yields263982263982instances spanning192192translation directions\. Because GRPO relies on within\-group reward variation, rollout groups with nearly identical rewards provide little training signal\. We therefore sampleGGtranslations for each instance using the RL rollout configuration, score them with our reward function, and retain instances whose group meanμx\\mu\_\{x\}and standard deviationσx\\sigma\_\{x\}satisfy0\.30<μx<0\.950\.30<\\mu\_\{x\}<0\.95andσx≥0\.05\\sigma\_\{x\}\\geq 0\.05\. The mean bounds remove instances with very low or high average rewards, while the standard\-deviation threshold removes groups with little reward variation\. This leaves3157231572instances, randomly split into3057230572training and10001000validation instances\. A detailed breakdown by translation direction is provided in Appendix[A](https://arxiv.org/html/2608.10812#A1)\.

#### SFT–RL checkpoint interpolation\.

LetθSFT\\theta\_\{\\mathrm\{SFT\}\}denote the parameters of MiLMMT\-46\-v0\.1 andθRL\\theta\_\{\\mathrm\{RL\}\}the parameters obtained after RL\. To reduce reward\-induced drift while retaining the gains from RL, we linearly interpolate the two checkpoints:

θα=α​θSFT\+\(1−α\)​θRL,α∈\[0,1\]\.\\theta\_\{\\alpha\}=\\alpha\\theta\_\{\\mathrm\{SFT\}\}\+\(1\-\\alpha\)\\theta\_\{\\mathrm\{RL\}\},\\qquad\\alpha\\in\[0,1\]\.Here,α\\alphacontrols the contribution of the SFT checkpoint, while1−α1\-\\alphacontrols that of the RL checkpoint\. The interpolation requires no additional training\. We refer toθRL\\theta\_\{\\mathrm\{RL\}\}as MiLMMT\-46\-v0\.1\-RL and the selected interpolated checkpoint as MiLMMT\-46\-v1\.0\.

## 3Experimental Setup

#### Datasets\.

We conduct experiments on4646languages spanning a broad linguistic spectrum, with detailed language information summarized in Table[4](https://arxiv.org/html/2608.10812#A2.T4)\. We evaluate multilingual translation performance on the FLORES\+NLLB Teamet al\.\([2024](https://arxiv.org/html/2608.10812#bib.bib6)\)and WMT24\+\+Deutschet al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib7)\)benchmarks\. For WMT24\+\+, we adopt the English source sentences and exclude those marked as low quality for reference\-free evaluation\.

#### Baselines\.

We compare against strong proprietary systems, including Google Translate, Gemini 2\.5/3 Pro, and GPT\-5, as well as the large\-scale multilingual NMT model NLLBTeamet al\.\([2022](https://arxiv.org/html/2608.10812#bib.bib21)\)\. In addition, we compare against several strong open\-source multilingual translation models:

- •Tower\-PlusReiet al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib12)\): Gemma2/Qwen2\.5\-based models for multilingual translation and general\-purpose tasks across2727languages\.
- •GemmaX2\-28Cuiet al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib13)\): Gemma2\-based models designed for multilingual machine translation across2828languages\.
- •Seed\-X\-Instruct/PPOChenget al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib14)\): Mistral\-based models trained with instruction finetuning and reinforcement learning for multilingual machine translation across2828languages\.
- •Hunyuan\-MT, HY\-MT1\.5, and HY\-MT2Zhenget al\.\([2025b](https://arxiv.org/html/2608.10812#bib.bib15),[a](https://arxiv.org/html/2608.10812#bib.bib16),[2026](https://arxiv.org/html/2608.10812#bib.bib17)\): Hunyuan\-based multilingual translation models across3333languages\.
- •TranslateGemmaFinkelsteinet al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib18)\): Gemma3\-based models for high\-quality translation across5555languages\.

#### Training and checkpoint interpolation configurations\.

We apply the same RL recipe at all three scales \(11B,44B, and1212B\), using theverl444[https://github\.com/verl\-project/verl](https://github.com/verl-project/verl)frameworkShenget al\.\([2025](https://arxiv.org/html/2608.10812#bib.bib24)\)with vLLM\-basedKwonet al\.\([2023](https://arxiv.org/html/2608.10812#bib.bib23)\)rollouts\. For each RL instance, we sampleG=8G\{=\}8candidates and regularize the policy toward the SFT referenceπref\\pi\_\{\\text\{ref\}\}with a low\-variance KL penalty\. After RL training, we interpolate the SFT and RL parameters to construct the final checkpoint\. For the MiLMMT\-46\-v1\.0 models, we set the interpolation coefficient toα=0\.5\\alpha=0\.5at all three scales\. We explore other values ofα\\alphain the interpolation analysis\. A complete list of hyper\-parameters is provided in Appendix[C](https://arxiv.org/html/2608.10812#A3)\.

#### Evaluation\.

We use XCOMET and COMETKiwi as reference\-free evaluators on both the FLORES\+ and WMT24\+\+ benchmarks\. Both are1010B\-parameter models with high correlation with human judgmentsFreitaget al\.\([2023](https://arxiv.org/html/2608.10812#bib.bib10)\)\. On FLORES\+, we additionally report reference\-based spBLEU and XCOMET scores\. For each baseline, we evaluate on the subset of languages shared with our4646\-language model\. Translations are generated using greedy decoding\.

Table 1:Translation performance on WMT24\+\+ and FLORES\+\. WMT24\+\+ QE cells report reference\-free XCOMET / COMETKiwi\. For FLORES\+, Ref\. cells report spBLEU / reference\-based XCOMET, while QE cells report reference\-free XCOMET / COMETKiwi\. The best and second\-best scores for each metric within each language\-coverage block are shown inboldandunderlined, respectively\. Detailed reference\-free and reference\-based results are provided in Appendix[D\.1](https://arxiv.org/html/2608.10812#A4.SS1)and Appendix[D\.2](https://arxiv.org/html/2608.10812#A4.SS2), respectively\.

## 4Experimental Results

### 4\.1Main Results

Table[1](https://arxiv.org/html/2608.10812#S3.T1)reports reference\-free results on WMT24\+\+ and both reference\-free and reference\-based results on FLORES\+\. Since the compared systems differ in language coverage, each block uses the language subset shared by its included systems\.

#### Effects of reference\-free post\-training\.

Within the full4646\-language block, MiLMMT\-46\-v1\.0 consistently improves over its supervised\-finetuned MiLMMT\-46\-v0\.1 counterpart across all three model scales\. Averaged over the three model scales, XCOMET and COMETKiwi scores on WMT24\+\+ improve by2\.752\.75and2\.442\.44points, respectively\. On FLORES\+, averaged over the four direction groups and three model scales, reference\-based XCOMET, reference\-free XCOMET, and COMETKiwi improve by1\.171\.17,1\.411\.41, and1\.171\.17points, while spBLEU decreases by1\.211\.21points\. These results show that reference\-free post\-training consistently improves learned quality metrics, with a modest reduction in lexical overlap\. Since spBLEU measures lexical overlap with a single reference translation and can penalize valid alternative wording, we view its modest decrease as less informative than the consistent gains in learned quality metrics\.

#### Full4646\-language comparison\.

MiLMMT\-46\-12B\-v1\.0 achieves the highest reference\-free XCOMET and COMETKiwi scores on WMT24\+\+ and across all four FLORES\+ direction groups, outperforming the evaluated proprietary systems, including Google Translate, Gemini 2\.5/3 Pro, and GPT\-5\. On reference\-based XCOMET, it ranks among the top two systems in three of the four FLORES\+ direction groups\. Among the smaller models, MiLMMT\-46\-1B\-v1\.0 outperforms TranslateGemma\-4B on every reported WMT24\+\+ and FLORES\+ metric despite using only one quarter as many parameters\. Although its spBLEU remains below Google Translate and Gemini 2\.5/3 Pro, MiLMMT\-46\-12B\-v1\.0 surpasses GPT\-5 in three of the four FLORES\+ direction groups and achieves a higher average across the four groups \(35\.1435\.14vs\.34\.9434\.94\)\. It also achieves a higher average spBLEU across the four groups than all evaluated external open\-source baselines, including NLLB\-54\.5B and TranslateGemma\-27B\.

#### Comparisons on shared language subsets\.

Across the2121\-,2626\-,2828\-, and3131\-language subsets, MiLMMT\-46\-12B\-v1\.0 ranks first in1515of the1616reference\-based XCOMET comparisons across the four FLORES\+ direction groups\. It also achieves the best reference\-free scores in nearly all comparisons against Tower\-Plus, Seed\-X, GemmaX2, and the HY\-MT series\. These results are not limited to the largest model\. MiLMMT\-46\-4B\-v1\.0 remains competitive with substantially larger models, including Tower\-Plus\-72B and HY\-MT2\-30B\-A3B\.

### 4\.2Analysis

![Refer to caption](https://arxiv.org/html/2608.10812v1/x1.png)Figure 1:Training and validation rewards during GRPO post\-training at the11B,44B, and1212B scales\. Validation rewards are evaluated every5050training steps\.![Refer to caption](https://arxiv.org/html/2608.10812v1/x2.png)Figure 2:Trade\-off between spBLEU and reference\-based XCOMET under SFT–RL checkpoint interpolation on FLORES\+, averaged over four translation\-direction groups:en→\\rightarrowxx,xx→\\rightarrowen,zh→\\rightarrowxx, andxx→\\rightarrowzh\. Each colored curve corresponds to one model scale, andα\\alphadenotes the SFT weight\. The MiLMMT\-46\-v0\.1\-RL and MiLMMT\-46\-v0\.1 checkpoints form the two endpoints of each curve\. Circled points indicate the selected MiLMMT\-46\-v1\.0 checkpoints atα=0\.5\\alpha=0\.5\. External systems are included for comparison\.#### Reference\-free post\-training gains persist across model scales\.

Figure[1](https://arxiv.org/html/2608.10812#S4.F1)shows that training and validation rewards increase early and then stabilize at all three model scales\. Comparing MiLMMT\-46\-v1\.0 with its SFT counterpart v0\.1 on WMT24\+\+, reference\-free XCOMET improves by2\.872\.87,2\.592\.59, and2\.792\.79points at the11B,44B, and1212B scales, respectively\. COMETKiwi shows the same trend\. These results indicate that reference\-free post\-training remains effective across model scales, including on top of the strongest1212B SFT model, rather than being limited to smaller models\.

#### Interpolation as a controllable quality trade\-off\.

Figure[2](https://arxiv.org/html/2608.10812#S4.F2)shows a consistent trade\-off between spBLEU and reference\-based XCOMET across all three model scales\. As the SFT weightα\\alphaincreases, the interpolated checkpoints move toward the SFT endpoint: spBLEU increases monotonically, while reference\-based XCOMET decreases\. The selectedα=0\.5\\alpha=0\.5checkpoints provide a favorable balance along these curves\. Relative to the RL endpoints, they recover2\.792\.79,3\.703\.70, and4\.214\.21spBLEU points at the11B,44B, and1212B scales, respectively, while reducing reference\-based XCOMET by only0\.530\.53,0\.560\.56, and0\.540\.54points\. Increasingα\\alphabeyond0\.50\.5yields diminishing returns in spBLEU\. For the1212B model, increasingα\\alphafrom0\.50\.5to0\.70\.7gains another1\.311\.31spBLEU points at a cost of0\.550\.55XCOMET, whereas increasing it from0\.70\.7to0\.90\.9gains only0\.370\.37spBLEU points at a further cost of0\.390\.39XCOMET\. We therefore fixα=0\.5\\alpha=0\.5across model scales as a simple operating point that recovers a substantial portion of the spBLEU lost during RL while retaining most of the improvement in reference\-based XCOMET\.

## 5On\-Policy Distillation Analysis

Table 2:On\-policy distillation from MiLMMT\-46\-12B\-v1\.0 into11B and44B students, averaged over4646languages\. The v1\.0 baseline is obtained through RL and checkpoint interpolation, and parenthetical labels indicate the student initialization\. WMT24\+\+ reports reference\-free XCOMET, while FLORES\+ reports spBLEU and reference\-based XCOMET\. RL \+ OPD adoptsλ=1\\lambda=1\. Full per\-group results and results for all tested values ofλ\\lambdaare provided in Appendix[E](https://arxiv.org/html/2608.10812#A5)\.We study whether on\-policy distillation \(OPD\) can transfer the gains of a post\-trained teacher to smaller students without running reinforcement learning separately at each model scale\. We use MiLMMT\-46\-12B\-v1\.0 as the teacher and train11B and44B students using the same reference\-free RL data described in Section[2](https://arxiv.org/html/2608.10812#S2)\. The students generate their own rollouts, and the teacher provides a distillation signal at the student\-induced states\.

Given a promptxx, a student rollouty∼πθ\(⋅∣x\)y\\sim\\pi\_\{\\theta\}\(\\cdot\\mid x\), and a statest=\(x,y<t\)s\_\{t\}=\(x,y\_\{<t\}\), OPD trains the student by minimizing the expected token\-level divergence between the student policy and a teacher policy:

ℒOPD\(θ\)=𝔼x,y∼πθ\[1\|y\|∑t=1\|y\|D\(πθ\(⋅∣st\),πϕ\(⋅∣st\),yt\)\]\\displaystyle\\mathcal\{L\}\_\{\\mathrm\{OPD\}\}\(\\theta\)=\\mathbb\{E\}\_\{x,\\,y\\sim\\pi\_\{\\theta\}\}\\\!\\left\[\\tfrac\{1\}\{\|y\|\}\\textstyle\\sum\_\{t=1\}^\{\|y\|\}D\\\!\\big\(\\pi\_\{\\theta\}\(\\cdot\\mid s\_\{t\}\),\\pi\_\{\\phi\}\(\\cdot\\mid s\_\{t\}\),y\_\{t\}\\big\)\\right\]

whereπϕ\\pi\_\{\\phi\}denotes the teacher policy andDDis a per\-token divergence signal\. This objective admits two implementations\. The distributional form, GKDAgarwalet al\.\([2024](https://arxiv.org/html/2608.10812#bib.bib27)\), definesDDas a distribution\-level KL and backpropagates through it directly\. The policy\-gradient form, PG\-OPDLu and Lab \([2025](https://arxiv.org/html/2608.10812#bib.bib26)\), estimates the divergence from student\-sampled tokens and optimizes the resulting signal as a reward through an RL update\.

We adopt PG\-OPD as our primary implementation to study OPD and RL under a unified objective\. Unless stated otherwise, OPD refers to the policy\-gradient form\. We report GKD only as a control experiment in Appendix[E](https://arxiv.org/html/2608.10812#A5)\. Specifically, PG\-OPD instantiates the per\-token divergence for a student\-sampled tokenyty\_\{t\}as

Dt=sg⁡\(log⁡πθ​\(yt∣st\)−log⁡πϕ​\(yt∣st\)\),D\_\{t\}=\\operatorname\{sg\}\\\!\\big\(\\log\\pi\_\{\\theta\}\(y\_\{t\}\\mid s\_\{t\}\)\-\\log\\pi\_\{\\phi\}\(y\_\{t\}\\mid s\_\{t\}\)\\big\),which is a single\-sample estimator of the reverse KL divergence from the student policy to the teacher policy\. SinceℒOPD\\mathcal\{L\}\_\{\\mathrm\{OPD\}\}averagesDtD\_\{t\}over student\-sampled tokens, minimizing it is equivalent to maximizing its negative\. We therefore usert=−Dtr\_\{t\}=\-D\_\{t\}as the per\-token reward and optimize it with the same GRPO objective described in Section[2](https://arxiv.org/html/2608.10812#S2)\. Here,sg⁡\(⋅\)\\operatorname\{sg\}\(\\cdot\)denotes the stop\-gradient operator, which prevents gradients from flowing through the reward computation while preserving its dependence on the teacher policy\. We denote the resulting policy\-gradient distillation loss byℒdistill\\mathcal\{L\}\_\{\\mathrm\{distill\}\}and the GRPO loss from Section[2](https://arxiv.org/html/2608.10812#S2)byℒpolicy\\mathcal\{L\}\_\{\\mathrm\{policy\}\}\.

We compare three training variants that differ in their objectives and student initialization\. OPD \(init\. v0\.1\) optimizesℒdistill\\mathcal\{L\}\_\{\\mathrm\{distill\}\}alone starting from MiLMMT\-46\-v0\.1\. RL\+OPD \(init\. v0\.1\) combines the GRPO and distillation losses:

ℒ=ℒpolicy\+λ​ℒdistill\\mathcal\{L\}=\\mathcal\{L\}\_\{\\mathrm\{policy\}\}\+\\lambda\\,\\mathcal\{L\}\_\{\\mathrm\{distill\}\}whereλ\\lambdacontrols the distillation weight\. OPD \(init\. v1\.0\) instead optimizesℒdistill\\mathcal\{L\}\_\{\\mathrm\{distill\}\}alone starting from the corresponding MiLMMT\-46\-v1\.0 student\.

#### Transferring post\-training gains through OPD\.

Table[2](https://arxiv.org/html/2608.10812#S5.T2)compares the distilled students with the corresponding MiLMMT\-46\-v1\.0 baselines\. OPD \(init\. v0\.1\) closely matches the44B baseline on FLORES\+, achieving34\.1734\.17versus33\.9633\.96spBLEU and90\.8290\.82versus90\.9190\.91reference\-based XCOMET, while the11B student remains slightly behind in reference\-based XCOMET \(85\.2285\.22versus85\.9485\.94\)\. Overall, OPD from the post\-trained1212B teacher approaches the performance of the corresponding MiLMMT\-46\-v1\.0 models, but does not consistently match or surpass them\.

#### Distillation weight controls the quality trade\-off\.

Varyingλ\\lambdain RL \+ OPD produces a trade\-off similar to that obtained through SFT–RL checkpoint interpolation in Section[4\.2](https://arxiv.org/html/2608.10812#S4.SS2)\. At44B, reducingλ\\lambdafrom11to0\.0010\.001increases reference\-based XCOMET from90\.9890\.98to91\.4791\.47while decreasing spBLEU from33\.9833\.98to30\.2030\.20, with intermediate values following the same trend\. Withλ=1\\lambda=1, RL \+ OPD remains close to the corresponding MiLMMT\-46\-v1\.0 baselines across both benchmarks and model scales\. Initializing OPD from MiLMMT\-46\-v1\.0 rather than MiLMMT\-46\-v0\.1 likewise has little effect\.

#### Overall finding\.

Across objectives, distillation weights, and student initializations, OPD and RL \+ OPD remain close to the quality frontier established by RL and checkpoint interpolation but do not extend it\. We therefore view OPD as a robustness finding and an alternative way to transfer post\-training gains to smaller models, rather than an improvement over RL and interpolation\.

## 6Conclusion

In this paper, we study reference\-free post\-training for multilingual machine translation with open large language models, analyzing GRPO, SFT–RL checkpoint interpolation, and on\-policy distillation across the11B,44B, and1212B model scales\. Experiments covering4646languages on WMT24\+\+ and FLORES\+ show that language\-gated quality estimation rewards consistently improve reference\-free XCOMET and COMETKiwi, as well as reference\-based XCOMET, although the resulting RL checkpoints exhibit lower spBLEU\. Building on MiLMMT\-46\-v0\.1, we develop MiLMMT\-46\-v1\.0 by interpolating the SFT and RL checkpoints, recovering a substantial portion of the spBLEU lost during RL while retaining most of the gains in neural quality metrics\. The resulting models compare favorably with strong recent open\-source baselines, and MiLMMT\-46\-12B\-v1\.0 achieves leading reference\-free results among the evaluated open and proprietary systems\. Our on\-policy distillation experiments further show that the gains of the post\-trained1212B teacher can be transferred to smaller students, although distillation does not improve on the trade\-off achieved by RL and checkpoint interpolation\. We hope our findings and released models support scalable and reference\-free post\-training for open multilingual translation systems\. In future work, we will explore more robust and human\-aligned reward models, extend reference\-free post\-training to multimodal multilingual translation, and study its behavior at larger model scales\.

## References

- R\. Agarwal, N\. Vieillard, Y\. Zhou, P\. Stanczyk, S\. Ramos Garea, M\. Geist, and O\. Bachem \(2024\)On\-policy distillation of language models: learning from self\-generated mistakes\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 21246–21263\.Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p4.1),[§5](https://arxiv.org/html/2608.10812#S5.p2.6)\.
- S\. Cheng, Y\. Bao, Q\. Cao, L\. Huang, L\. Kang, Z\. Liu, Y\. Lu, W\. Zhu, J\. Chen, Z\. Huang, T\. Li, Y\. Li, H\. Lin, S\. Liu, N\. Peng, S\. She, L\. Xu, N\. Xu, S\. Yang, R\. Yu, Y\. Yu, L\. Zou, H\. Li, L\. Lu, Y\. Wang, and Y\. Wu \(2025\)Seed\-x: building strong multilingual translation llm with 7b parameters\.External Links:2507\.13618,[Link](https://arxiv.org/abs/2507.13618)Cited by:[3rd item](https://arxiv.org/html/2608.10812#S3.I1.i3.p1.1)\.
- M\. Cui, P\. Gao, W\. Liu, J\. Luan, and B\. Wang \(2025\)Multilingual machine translation with open large language models at practical scale: an empirical study\.InProceedings of the 2025 Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics: Human Language Technologies \(Volume 1: Long Papers\),L\. Chiruzzo, A\. Ritter, and L\. Wang \(Eds\.\),Albuquerque, New Mexico,pp\. 5420–5443\.External Links:[Link](https://aclanthology.org/2025.naacl-long.280/),[Document](https://dx.doi.org/10.18653/v1/2025.naacl-long.280),ISBN 979\-8\-89176\-189\-6Cited by:[2nd item](https://arxiv.org/html/2608.10812#S3.I1.i2.p1.1)\.
- D\. Deutsch, E\. Briakou, I\. R\. Caswell, M\. Finkelstein, R\. Galor, J\. Juraska, G\. Kovacs, A\. Lui, R\. Rei, J\. Riesa, S\. Rijhwani, P\. Riley, E\. Salesky, F\. Trabelsi, S\. Winkler, B\. Zhang, and M\. Freitag \(2025\)WMT24\+\+: expanding the language coverage of WMT24 to 55 languages & dialects\.InFindings of the Association for Computational Linguistics: ACL 2025,W\. Che, J\. Nabende, E\. Shutova, and M\. T\. Pilehvar \(Eds\.\),Vienna, Austria,pp\. 12257–12284\.External Links:[Link](https://aclanthology.org/2025.findings-acl.634/),[Document](https://dx.doi.org/10.18653/v1/2025.findings-acl.634),ISBN 979\-8\-89176\-256\-5Cited by:[§3](https://arxiv.org/html/2608.10812#S3.SS0.SSS0.Px1.p1.1)\.
- M\. Fedorova, N\. Arefyev, M\. Buljan, J\. Helcl, S\. Oepen, E\. Rønningstad, and Y\. Scherrer \(2026\)OpenLID\-v3: improving the precision of closely related language identification – an experience report\.InProceedings of the 13th Workshop on NLP for Similar Languages, Varieties and Dialects,Y\. Scherrer, N\. Aepli, V\. Blaschke, T\. Jauhiainen, N\. Ljubešić, P\. Nakov, J\. Tiedemann, and M\. Zampieri \(Eds\.\),Rabat, Morocco,pp\. 275–292\.External Links:[Link](https://aclanthology.org/2026.vardial-1.23/),[Document](https://dx.doi.org/10.18653/v1/2026.vardial-1.23)Cited by:[§2](https://arxiv.org/html/2608.10812#S2.SS0.SSS0.Px2.p1.8)\.
- M\. Finkelstein, I\. Caswell, T\. Domhan, J\. Peter, J\. Juraska, P\. Riley, D\. Deutsch, C\. Dilanni, C\. Cherry, E\. Briakou, E\. Nielsen, J\. Luo, K\. Black, R\. Mullins, S\. Agrawal, W\. Xu, E\. Kats, S\. Jaskiewicz, M\. Freitag, and D\. Vilar \(2026\)TranslateGemma technical report\.External Links:2601\.09012,[Link](https://arxiv.org/abs/2601.09012)Cited by:[5th item](https://arxiv.org/html/2608.10812#S3.I1.i5.p1.1)\.
- M\. Freitag, N\. Mathur, C\. Lo, E\. Avramidis, R\. Rei, B\. Thompson, T\. Kocmi, F\. Blain, D\. Deutsch, C\. Stewart, C\. Zerva, S\. Castilho, A\. Lavie, and G\. Foster \(2023\)Results of WMT23 metrics shared task: metrics might be guilty but references are not innocent\.InProceedings of the Eighth Conference on Machine Translation,P\. Koehn, B\. Haddow, T\. Kocmi, and C\. Monz \(Eds\.\),Singapore,pp\. 578–628\.External Links:[Link](https://aclanthology.org/2023.wmt-1.51/),[Document](https://dx.doi.org/10.18653/v1/2023.wmt-1.51)Cited by:[§3](https://arxiv.org/html/2608.10812#S3.SS0.SSS0.Px4.p1.2)\.
- Gemma Team \(2026\)Gemma 4 technical report\.External Links:2607\.02770,[Link](https://arxiv.org/abs/2607.02770)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p1.1)\.
- N\. M\. Guerreiro, R\. Rei, D\. v\. Stigt, L\. Coheur, P\. Colombo, and A\. F\. T\. Martins \(2024\)XCOMET: transparent machine translation evaluation through fine\-grained error detection\.Transactions of the Association for Computational Linguistics12,pp\. 979–995\.External Links:[Link](https://aclanthology.org/2024.tacl-1.54/),[Document](https://dx.doi.org/10.1162/tacl%5Fa%5F00683)Cited by:[§2](https://arxiv.org/html/2608.10812#S2.SS0.SSS0.Px2.p1.8)\.
- P\. Joshi, S\. Santy, A\. Budhiraja, K\. Bali, and M\. Choudhury \(2020\)The state and fate of linguistic diversity and inclusion in the NLP world\.InProceedings of the 58th Annual Meeting of the Association for Computational Linguistics,D\. Jurafsky, J\. Chai, N\. Schluter, and J\. Tetreault \(Eds\.\),Online,pp\. 6282–6293\.External Links:[Link](https://aclanthology.org/2020.acl-main.560/),[Document](https://dx.doi.org/10.18653/v1/2020.acl-main.560)Cited by:[Table 4](https://arxiv.org/html/2608.10812#A2.T4)\.
- W\. Kwon, Z\. Li, S\. Zhuang, Y\. Sheng, L\. Zheng, C\. H\. Yu, J\. E\. Gonzalez, H\. Zhang, and I\. Stoica \(2023\)Efficient memory management for large language model serving with pagedattention\.InProceedings of the ACM SIGOPS 29th Symposium on Operating Systems Principles,Cited by:[§3](https://arxiv.org/html/2608.10812#S3.SS0.SSS0.Px3.p1.7)\.
- Y\. Liu, S\. Ouyang, H\. R\. Yatish, and L\. Li \(2026\)Mending the holes: mitigating reward hacking in reinforcement learning for multilingual translation\.InFindings of the Association for Computational Linguistics: ACL 2026,M\. Liakata, V\. P\. Moreira, J\. Zhang, and D\. Jurgens \(Eds\.\),San Diego, California, United States,pp\. 33699–33723\.External Links:[Link](https://aclanthology.org/2026.findings-acl.1682/),[Document](https://dx.doi.org/10.18653/v1/2026.findings-acl.1682)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p3.1),[§2](https://arxiv.org/html/2608.10812#S2.SS0.SSS0.Px2.p1.9)\.
- K\. Lu and T\. M\. Lab \(2025\)On\-policy distillation\.Thinking Machines Lab: Connectionism\.Note:https://thinkingmachines\.ai/blog/on\-policy\-distillationExternal Links:[Document](https://dx.doi.org/10.64434/tml.20251026)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p4.1),[§5](https://arxiv.org/html/2608.10812#S5.p2.6)\.
- NLLB Team, M\. R\. Costa\-jussà, J\. Cross, O\. Çelebi, M\. Elbayad, K\. Heafield, K\. Heffernan, E\. Kalbassi, J\. Lam, D\. Licht, J\. Maillard, A\. Sun, S\. Wang, G\. Wenzek, A\. Youngblood, B\. Akula, L\. Barrault, G\. M\. Gonzalez, P\. Hansanti, J\. Hoffman, S\. Jarrett, K\. R\. Sadagopan, D\. Rowe, S\. Spruit, C\. Tran, P\. Andrews, N\. F\. Ayan, S\. Bhosale, S\. Edunov, A\. Fan, C\. Gao, V\. Goswami, F\. Guzmán, P\. Koehn, A\. Mourachko, C\. Ropers, S\. Saleem, H\. Schwenk, and J\. Wang \(2024\)Scaling neural machine translation to 200 languages\.Nature630\(8018\),pp\. 841–846\.External Links:ISSN 1476\-4687,[Document](https://dx.doi.org/10.1038/s41586-024-07335-x),[Link](https://doi.org/10.1038/s41586-024-07335-x)Cited by:[§3](https://arxiv.org/html/2608.10812#S3.SS0.SSS0.Px1.p1.1)\.
- Qwen Team \(2025\)Qwen3 technical report\.External Links:2505\.09388,[Link](https://arxiv.org/abs/2505.09388)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p1.1)\.
- R\. Rei, N\. M\. Guerreiro, J\. Pombal, J\. Alves, P\. Teixeirinha, A\. Farajian, and A\. F\. T\. Martins \(2025\)Tower\+: bridging generality and translation specialization in multilingual llms\.External Links:2506\.17080,[Link](https://arxiv.org/abs/2506.17080)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p3.1),[1st item](https://arxiv.org/html/2608.10812#S3.I1.i1.p1.1)\.
- R\. Rei, N\. M\. Guerreiro, J\. Pombal, D\. van Stigt, M\. Treviso, L\. Coheur, J\. G\. C\. de Souza, and A\. F\. T\. Martins \(2023\)Scaling up CometKiwi: unbabel\-IST 2023 submission for the quality estimation shared task\.InProceedings of the Eighth Conference on Machine Translation,P\. Koehn, B\. Haddow, T\. Kocmi, and C\. Monz \(Eds\.\),Singapore,pp\. 841–848\.External Links:[Link](https://aclanthology.org/2023.wmt-1.73/),[Document](https://dx.doi.org/10.18653/v1/2023.wmt-1.73)Cited by:[§2](https://arxiv.org/html/2608.10812#S2.SS0.SSS0.Px2.p1.8)\.
- Y\. Shang, P\. Gao, W\. Liu, J\. Luan, and J\. Su \(2026\)Scaling model and data for multilingual machine translation with open large language models\.External Links:2602\.11961,[Link](https://arxiv.org/abs/2602.11961)Cited by:[Appendix A](https://arxiv.org/html/2608.10812#A1.p1.4),[§1](https://arxiv.org/html/2608.10812#S1.p1.1)\.
- Z\. Shao, P\. Wang, Q\. Zhu, R\. Xu, J\. Song, X\. Bi, H\. Zhang, M\. Zhang, Y\. K\. Li, Y\. Wu, and D\. Guo \(2024\)DeepSeekMath: pushing the limits of mathematical reasoning in open language models\.External Links:2402\.03300,[Link](https://arxiv.org/abs/2402.03300)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p3.1)\.
- G\. Sheng, C\. Zhang, Z\. Ye, X\. Wu, W\. Zhang, R\. Zhang, Y\. Peng, H\. Lin, and C\. Wu \(2025\)HybridFlow: a flexible and efficient rlhf framework\.InProceedings of the Twentieth European Conference on Computer Systems,pp\. 1279–1297\.Cited by:[§3](https://arxiv.org/html/2608.10812#S3.SS0.SSS0.Px3.p1.7)\.
- N\. Team, M\. R\. Costa\-jussà, J\. Cross, O\. Çelebi, M\. Elbayad, K\. Heafield, K\. Heffernan, E\. Kalbassi, J\. Lam, D\. Licht, J\. Maillard, A\. Sun, S\. Wang, G\. Wenzek, A\. Youngblood, B\. Akula, L\. Barrault, G\. M\. Gonzalez, P\. Hansanti, J\. Hoffman, S\. Jarrett, K\. R\. Sadagopan, D\. Rowe, S\. Spruit, C\. Tran, P\. Andrews, N\. F\. Ayan, S\. Bhosale, S\. Edunov, A\. Fan, C\. Gao, V\. Goswami, F\. Guzmán, P\. Koehn, A\. Mourachko, C\. Ropers, S\. Saleem, H\. Schwenk, and J\. Wang \(2022\)No language left behind: scaling human\-centered machine translation\.External Links:2207\.04672,[Link](https://arxiv.org/abs/2207.04672)Cited by:[§3](https://arxiv.org/html/2608.10812#S3.SS0.SSS0.Px2.p1.1)\.
- M\. Zheng, Z\. Li, T\. Chen, B\. Lv, M\. Sun, M\. Song, J\. Song, H\. Huang, D\. Wu, H\. Wang, Y\. Song, Y\. Chen, and G\. Zhang \(2026\)Hy\-mt2: a family of fast, efficient and powerful multilingual translation models in the wild\.External Links:2605\.22064,[Link](https://arxiv.org/abs/2605.22064)Cited by:[§1](https://arxiv.org/html/2608.10812#S1.p3.1),[4th item](https://arxiv.org/html/2608.10812#S3.I1.i4.p1.1)\.
- M\. Zheng, Z\. Li, T\. Chen, M\. Song, and D\. Wang \(2025a\)HY\-mt1\.5 technical report\.External Links:2512\.24092,[Link](https://arxiv.org/abs/2512.24092)Cited by:[4th item](https://arxiv.org/html/2608.10812#S3.I1.i4.p1.1)\.
- M\. Zheng, Z\. Li, B\. Qu, M\. Song, Y\. Du, M\. Sun, and D\. Wang \(2025b\)Hunyuan\-mt technical report\.External Links:2509\.05209,[Link](https://arxiv.org/abs/2509.05209)Cited by:[4th item](https://arxiv.org/html/2608.10812#S3.I1.i4.p1.1)\.

## Appendix ADistribution of RL Training Data

After effective\-data selection, the retained pool comprises3157231572source sentences \(3057230572for training and10001000held out for validation\) across192192translation directions\. Table[3](https://arxiv.org/html/2608.10812#A1.T3)reports the number of sentences per direction, following the language\-code convention ofShanget al\.\([2026](https://arxiv.org/html/2608.10812#bib.bib5)\)\.

Table 3:Number of instances for each translation direction in the RL training data, sorted by training count\. All192192directions are shown \(30,57230\{,\}572training /1,0001\{,\}000validation in total\)\.
## Appendix BLanguage Details

Table 4:4646languages supported by our model\. The resource of each language is determined according to the taxonomy classes byJoshiet al\.\([2020](https://arxiv.org/html/2608.10812#bib.bib11)\)\.
## Appendix CTraining Hyper\-parameters

Table[5](https://arxiv.org/html/2608.10812#A3.T5)lists the complete set of GRPO hyper\-parameters, shared across the11B,44B, and1212B models\.

Table 5:GRPO training hyper\-parameters, shared across all three model scales\.
## Appendix DDetailed Evaluation Results

### D\.1Reference\-Free Results

Tables[6](https://arxiv.org/html/2608.10812#A4.T6)–[11](https://arxiv.org/html/2608.10812#A4.T11)report the per\-direction reference\-free results \(XCOMET / COMETKiwi\)\.

Table 6:Reference\-free evaluation results \(XCOMET / COMETKiwi\) of baseline models \(Part I\) on the WMT24\+\+ benchmark\.Table 7:Reference\-free evaluation results \(XCOMET / COMETKiwi\) of baseline models \(Part II\) and MiLMMT models on the WMT24\+\+ benchmark\.Table 8:Reference\-free English\-centric evaluation results \(XCOMET / COMETKiwi\) of baseline models \(Part I\) on the FLORES\+ benchmark\.Table 9:Reference\-free English\-centric evaluation results \(XCOMET / COMETKiwi\) of baseline models \(Part II\) and MiLMMT models on the FLORES\+ benchmark\.Table 10:Reference\-free Chinese\-centric evaluation results \(XCOMET / COMETKiwi\) of baseline models \(Part I\) on the FLORES\+ benchmark\.Table 11:Reference\-free Chinese\-centric evaluation results \(XCOMET / COMETKiwi\) of baseline models \(Part II\) and MiLMMT models on the FLORES\+ benchmark\.
### D\.2Reference\-Based Results

Tables[12](https://arxiv.org/html/2608.10812#A4.T12)–[15](https://arxiv.org/html/2608.10812#A4.T15)report the per\-direction reference\-based FLORES\+ results \(spBLEU / XCOMET\)\.

Table 12:Reference\-based English\-centric evaluation results \(spBLEU / XCOMET\) of baseline models \(Part I\) on the FLORES\+ benchmark\.Table 13:Reference\-based English\-centric evaluation results \(spBLEU / XCOMET\) of baseline models \(Part II\) and MiLMMT models on the FLORES\+ benchmark\.Table 14:Reference\-based Chinese\-centric evaluation results \(spBLEU / XCOMET\) of baseline models \(Part I\) on the FLORES\+ benchmark\.Table 15:Reference\-based Chinese\-centric evaluation results \(spBLEU / XCOMET\) of baseline models \(Part II\) and MiLMMT models on the FLORES\+ benchmark\.

## Appendix EFull On\-Policy Distillation Results

#### OPD Training Details\.

We adopt optimization settings similar to those used for GRPO in our OPD experiments, including a learning rate of1×10−61\\times 10^\{\-6\}, a prompt batch size of128128, a PPO mini\-batch size of128128, three training epochs, and maximum prompt and response lengths of40964096tokens each\. For RL\+OPD, we use the same task reward and group sizeG=8G=8as in GRPO\. Pure OPD does not use the task reward and instead samples one student trajectory per prompt\. We use the fixed MiLMMT\-46\-12B\-v1\.0 model as the teacher and thek1k\_\{1\}estimator for PG\-OPD\. For the GKD control, we use a top\-128128approximation to the forward KL divergence\.

#### Full Results\.

Table[16](https://arxiv.org/html/2608.10812#A5.T16)reports the complete OPD and RL\+OPD results underlying Section[5](https://arxiv.org/html/2608.10812#S5)and Table[2](https://arxiv.org/html/2608.10812#S5.T2), as4646\-language macro\-averages\. All students are distilled from the fixed MiLMMT\-46\-12B\-v1\.0 teacher\.*RL\+OPD*adds the QE\-based task reward with distillation weightλ\\lambda;*OPD*rows use only the distillation loss\. Unless noted, the student is initialized from v0\.1;*init\. v1\.0*rows are initialized from the interpolated checkpoint instead\. WMT24\+\+ cells report reference\-free XCOMET / COMETKiwi; each FLORES\+ direction group reports spBLEU / reference\-based XCOMET / reference\-free XCOMET / COMETKiwi\.

Table 16:Full on\-policy distillation results as4646\-language\. All students are distilled from the fixed MiLMMT\-46\-12B\-v1\.0 teacher\. WMT24\+\+ cells report reference\-free XCOMET / COMETKiwi; each FLORES\+ cell reports spBLEU / reference\-based XCOMET / reference\-free XCOMET / COMETKiwi\.