When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop

# When and How Human Curation Backfires: Preference Alignment under Multi-Model Self-Consuming Loop
Source: [https://arxiv.org/html/2605.29267](https://arxiv.org/html/2605.29267)
###### Abstract

Foundation models are increasingly trained on synthetic data generated by prior model iterations rather than exclusively on real data\. Thisself\-consumingtraining paradigm can lead to model collapse, divergence, or bias amplification\. Recent work\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\)shows that incorporating human curation into the loop can steer a self\-consuming model toward human\-aligned behavior, but these analyses focus on a single,isolatedmodel that solely consumes its own outputs\. In practice, however, models often interact and train on input–output pairs produced by other models\. This paper studies self\-consuming training in themulti\-modelregime\. We first formalize a framework for interacting self\-consuming models and characterize when the resulting dynamical system converges to a stable point\. We then examine how human curation of one model affects its own alignment \(self\-influence\) and how such effects propagate to other models \(cross\-influence\)\. Unlike isolated settings where human curation always enhances model alignment, we show that cross\-model interactions can dampen or even invert this effect, ultimately degrading long\-term alignment\.

Machine Learning, ICML

## 1Introduction

Modern foundation models are no longer just consumers of data; they have become large\-scale producers whose outputs circulate throughout the broader data ecosystem, appearing on the web, entering annotation pipelines, and being incorporated into downstream finetuning datasets\. As a result, synthetic \(often curated\) data is routinely mixed with real data during training\. This creates aself\-consuming loopin which models are iteratively updated on synthetic data generated by earlier model iterations\. For example, in language, synthetic instruction data generated by one LLM is used to finetune another model, reducing reliance on costly human annotation\(Taoriet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib27)\)\. In vision, diffusion models generate labeled images that are merged into large\-scale datasets for training or finetuning recognition models\(Shumailovet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib11)\)\. In multimodal settings, auto\-captioning models produce synthetic text that accompanies scraped images, and these image–caption pairs are later reused when training image–text models at scale\(Yuet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib28)\)\.

Motivated by this, a growing body of work has examined how models evolve under self\-consuming training loops\(Shumailovet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib11); Gao and Li,[2025](https://arxiv.org/html/2605.29267#bib.bib8); Bertrandet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib9); Caiet al\.,[2026](https://arxiv.org/html/2605.29267#bib.bib2)\)\. Prior work has shown, both theoretically and empirically, that when model\-generated content is reused to train successive models, such loops can induce degenerative dynamics and lead to unintended outcomes, including model collapse\(Shumailovet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib11); Bertrandet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib9); Fuet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib13)\), divergence\(Shumailovet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib11)\), and bias amplification\(Wanget al\.,[2026](https://arxiv.org/html/2605.29267#bib.bib18); Wyllieet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib20); Xie and Zhang,[2024](https://arxiv.org/html/2605.29267#bib.bib1)\)\. Recent works\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6); Zhaoet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib15)\)further explore the role of human curation in this process and show that when synthetic data is curated by humans according to their preferences at each iteration before being incorporated into the training set, models trained under self\-consuming loops gradually align with human preferences and their outputs can converge to a distribution that maximizes human expected reward\. Conversely, when human curation is noisy or adversarial, this alignment process may be disrupted\(Wei and Zhang,[2025](https://arxiv.org/html/2605.29267#bib.bib10)\)\.

However, existing work on self\-consuming models is largely limited to a single,isolatedmodel that consumes only its own outputs, whereas real\-world data ecosystems are inherentlymulti\-model\(Pressmanet al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib23)\)\. In web\-scale corpora \(e\.g\., LAION\-5B\(Schuhmannet al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib19)\)\), model\-generated data can be scraped and incorporated into future training pipelines, effectively coupling different models through recycled data\(Alemohammadet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib12)\)\. In such systems, updating one model reshapes the data distribution used to train others, which may then feed back into the first, forming a network of implicit interactions\.

To the best of our knowledge, only a few studies consider multi\-model self\-consuming systems\(Huet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib7); Gao and Li,[2025](https://arxiv.org/html/2605.29267#bib.bib8)\)\. However, these works are either empirical—highlighting new collapse patterns under multi\-model recursive reuse\(Huet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib7)\)—or theoretically tractable only under highly simplified distributional assumptions and update rules\(Gao and Li,[2025](https://arxiv.org/html/2605.29267#bib.bib8)\)\. As a result, we still lack a unified theoretical framework to address several practically motivated questions: How does a multi\-model ecosystem evolve under iterative self\-consuming loops? Under what conditions does such a system converge to a stable state? How does human curation applied to one model influence its own alignment and that of other models in the long run?

In this paper, we study a multi\-model setting in which models iteratively update on mixtures of real data and synthetic data generated and curated by themselves and by other models\. We first formalize an analytical framework for interacting self\-consuming models and characterize conditions under which the system converges to a stable point\. Given convergence, we then analyze the impact of human curation on the system’s long\-term alignment\. Specifically, we conduct local sensitivity analyses to quantify how increasing the fraction of human\-curated synthetic data for one model affects its own alignment \(self\-influence\) and how these effects propagate to other models \(cross\-influence\)\. Our results highlight a key difference from the single\-model setting: whereas increasing human curation consistently improves long\-term alignment in isolated self\-consuming loops\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\), its effect becomes non\-monotonic in the multi\-model regime\. Cross\-model interactions may amplify, dampen, or even reverse the impact of human curation, and our analysis identifies conditions under which human curation can negatively affect alignment in the long run\. Beyond these local effects, we also quantify the global alignment gap by comparing mixed real–synthetic training to purely real\-data training\. We discuss more related works in Appendix[A](https://arxiv.org/html/2605.29267#A1)and summarize key contributions as follows:

- •We formalize a framework for interacting multi\-model self\-consuming systems, which captures a wide range of cross\-model interactions and incorporates human preference feedback into the self\-consuming loop, without relying on stylized distributional assumptions \(Section[2](https://arxiv.org/html/2605.29267#S2)\)\.
- •We provide a rigorous analysis to identify conditions under which the multi\-model system converges to a stable point, and examine how the real\-data ratio contributes to stability under cross\-model interactions \(Section[3](https://arxiv.org/html/2605.29267#S3)\)\.
- •We quantify the long\-term impact of human curation on the model alignment\. The results are interpretable and decompose the effects intoself\-influence\(the model’s direct response to curation\) andcross\-influence\(the influence propagated through cross\-model interactions\)\. This allows us to identify conditions under which human curation improves, or degrades, long\-term alignment \(Section[4](https://arxiv.org/html/2605.29267#S4)\)\.
- •We validate our theoretical findings through real experiments, demonstrating that increasing human curation does not necessarily improve model alignment and the non\-monotonic behavior aligns with theorems \(Section[5](https://arxiv.org/html/2605.29267#S5)\)\.

## 2Problem Formulation

Consider a multi\-model ecosystem in which each model is iteratively updated on a mixture of real data and synthetic data generated by itself or by other models\. For simplicity, we focus on two models that, at each roundtt, are parametrized byθt∈Θ\\theta\_\{t\}\\in\\Thetaandϕt∈Φ\\phi\_\{t\}\\in\\Phirespectively, where bothΘ,Φ\\Theta,\\Phiare closed convex sets\. Letx∈𝒳x\\in\\mathcal\{X\}andy∈𝒴y\\in\\mathcal\{Y\}denote the outputs of modelsθt\\theta\_\{t\}andϕt\\phi\_\{t\}, respectively\. We assume that the input \(resp\. output\) data type ofθt\\theta\_\{t\}matches the output \(resp\. input\) data type ofϕt\\phi\_\{t\}\. Under this assumption, the corresponding conditional data distributions are defined as

x∼pθt​\(x\|y\)​and​y∼qϕt​\(y\|x\)x\\sim p\_\{\\theta\_\{t\}\}\(x\|y\)~\\text\{ and \}~y\\sim q\_\{\\phi\_\{t\}\}\(y\|x\)This formulation captures a wide range of cross\-model interactions, including interactions among generative models of the same modality \(e\.g\., text models whose outputs are reused as training data for other text models\), interactions across different modalities \(e\.g\., image\-to\-text and text\-to\-image models that feed into one another\), and interactions between generative and discriminative models \(e\.g\., classifiers that predict category labels and generative models that produce images or text conditioned on those categories\)\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/x1.png)

Figure 1:\(A1\) and \(A2\) illustrate the framework of asingleself\-consuming model and how to generate mixture data in each round\. Real data helps to stabilize model updating \(b\)\(Bertrandet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib9)\), and human curation benefits the reward improving \(a\)\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\)\. \(B1\) presents our multi\-model interaction framework\. Different models may be updated in different orders\. \(B2\) details the one\-round iteration between two models\. Under model interaction, multiple preferences exist within the system \(e\), and one model may use data generated/curated by other models for further training \(cd\), thus affecting its own preference alignment\. It remains unclear how interactions affect each model under multi\-model self\-consuming learning \(f\)\.Sources of training data\.To maintain strong performance and adapt to evolving environments, both modelsθt\\theta\_\{t\}andϕt\\phi\_\{t\}are repeatedly updated using their most recently acquired training datasets,𝒟t\+1θ\\mathcal\{D\}^\{\\theta\}\_\{t\+1\}and𝒟t\+1ϕ\\mathcal\{D\}^\{\\phi\}\_\{t\+1\}, respectively\. These datasets are typically drawn from multiple sources:

1. 1\.Real dataℛθ,ℛϕ\\mathcal\{R\}^\{\\theta\},\\mathcal\{R\}^\{\\phi\}whose distributions are fixed\.
2. 2\.Self\-consuming synthetic data𝒮t\+1θ,𝒮t\+1ϕ\\mathcal\{S\}\_\{t\+1\}^\{\\theta\},\\mathcal\{S\}\_\{t\+1\}^\{\\phi\}, consisting of samples generated either by the model itself or by other models\. LetPt\+1x,yP\_\{t\+1\}^\{x,y\}andQt\+1x,yQ\_\{t\+1\}^\{x,y\}denote the distributions over data pairs\(x,y\)\(x,y\)used to obtain the updated modelsθt\+1\\theta\_\{t\+1\},ϕt\+1\\phi\_\{t\+1\}fromθt\\theta\_\{t\}andϕt\\phi\_\{t\}, respectively\. Outputs generated by the models atttmay be incorporated into the training data of future model generations\. In particular, modelθt\\theta\_\{t\}can generate synthetic samples of the form \{\(x′,y′\):y′∼Pty,x′∼pθt​\(x\|y′\)\},\\\{\(x^\{\\prime\},y^\{\\prime\}\):y^\{\\prime\}\\sim P\_\{t\}^\{y\},x^\{\\prime\}\\sim p\_\{\\theta\_\{t\}\}\(x\|y^\{\\prime\}\)\\\},\(1\)while modelϕt\\phi\_\{t\}can generate samples \{\(x′,y′\):x′∼Qtx,y′∼qϕt​\(y\|x′\)\},\\\{\(x^\{\\prime\},y^\{\\prime\}\):x^\{\\prime\}\\sim Q\_\{t\}^\{x\},y^\{\\prime\}\\sim q\_\{\\phi\_\{t\}\}\(y\|x^\{\\prime\}\)\\\},\(2\)wherePtyP\_\{t\}^\{y\}andQtxQ\_\{t\}^\{x\}denote the corresponding marginals ofPtx,yP\_\{t\}^\{x,y\}andQtx,yQ\_\{t\}^\{x,y\}\. To capture both self\-model and cross\-model influence, we allow𝒮t\+1θ,𝒮t\+1ϕ\\mathcal\{S\}\_\{t\+1\}^\{\\theta\},\\mathcal\{S\}\_\{t\+1\}^\{\\phi\}to include samples generated by either model, with cross\-model\-generated fractionsλθϕ\\lambda\_\{\\theta\}^\{\\phi\}andλϕθ\\lambda\_\{\\phi\}^\{\\theta\}, respectively, and self\-generated fractions1−λθϕ1\-\\lambda\_\{\\theta\}^\{\\phi\}and1−λϕθ1\-\\lambda\_\{\\phi\}^\{\\theta\}, respectively\.
3. 3\.Human\-curated synthetic dataℋt\+1θ,ℋt\+1ϕ\\mathcal\{H\}\_\{t\+1\}^\{\\theta\},\\mathcal\{H\}\_\{t\+1\}^\{\\phi\}\. To align models with human preferences, synthetic data is often refined using human feedback and curated by users based on their preferences\. This process is commonly modeled using the generalized Bradley\-Terry model\(Bradley and Terry,[1952](https://arxiv.org/html/2605.29267#bib.bib32); Luce and others,[1959](https://arxiv.org/html/2605.29267#bib.bib50)\)or Direct Preference Optimization\(Rafailovet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib33)\)\. For example, givenKKsamples𝒪=\{o1,⋯,oK\}\\mathcal\{O\}=\\\{o\_\{1\},\\cdots,o\_\{K\}\\\},ok∼pθt​\(x\|i\)o\_\{k\}\\sim p\_\{\\theta\_\{t\}\}\(x\|i\)\(orpϕt​\(y\|i\)p\_\{\\phi\_\{t\}\}\(y\|i\)\) generated by modelθt\\theta\_\{t\}\(orϕt\\phi\_\{t\}\) for a given inputi∼Ptyi\\sim P\_\{t\}^\{y\}\(orQtxQ\_\{t\}^\{x\}\), the probability that a sampleo^∈𝒪\\hat\{o\}\\in\\mathcal\{O\}is selected by the user under generalized Bradley\-Terry model is: ℙ​\(o^=os\|i,𝒪\)=er​\(i,os\)∑k=1Ker​\(i,ok\)\\displaystyle\\mathbb\{P\}\(\\hat\{o\}=o\_\{s\}\|i,\\mathcal\{O\}\)=\\frac\{e^\{r\(i,o\_\{s\}\)\}\}\{\\sum\_\{k=1\}^\{K\}e^\{r\(i,o\_\{k\}\)\}\}\(3\)whererris the underlying reward function capturing the user’s preference for one sample over another\. The curated datasetsℋt\+1θ,ℋt\+1ϕ\\mathcal\{H\}\_\{t\+1\}^\{\\theta\},\\mathcal\{H\}\_\{t\+1\}^\{\\phi\}also contain samples curated from either model, and we assume that their cross\-model data fractions match those of𝒮t\+1θ,𝒮t\+1ϕ\\mathcal\{S\}\_\{t\+1\}^\{\\theta\},\\mathcal\{S\}\_\{t\+1\}^\{\\phi\}\.

For each modelj∈\{θ,ϕ\}j\\in\\\{\\theta,\\phi\\\}, the distribution underlying the training data𝒟t\+1j\\mathcal\{D\}^\{j\}\_\{t\+1\}is a mixture overℛt\+1j,𝒮t\+1j,ℋt\+1j\\mathcal\{R\}\_\{t\+1\}^\{j\},\\mathcal\{S\}\_\{t\+1\}^\{j\},\\mathcal\{H\}\_\{t\+1\}^\{j\}\. The correspondingmixing weightsare denoted byλℛj,λ𝒮j,λℋj\\lambda\_\{\\mathcal\{R\}\}^\{j\},\\lambda\_\{\\mathcal\{S\}\}^\{j\},\\lambda\_\{\\mathcal\{H\}\}^\{j\}, respectively, and satisfyλℛj\+λ𝒮j\+λℋj=1\\lambda\_\{\\mathcal\{R\}\}^\{j\}\+\\lambda\_\{\\mathcal\{S\}\}^\{j\}\+\\lambda\_\{\\mathcal\{H\}\}^\{j\}=1\.

Iterative training loop\.Since both self\-consuming and human\-curated synthetic data depend onθ,ϕ\\theta,\\phi, we denote the resulting training data distributions byPt\+1x,y:=P​\(θ,ϕ\)P\_\{t\+1\}^\{x,y\}:=P\(\\theta,\\phi\)andQt\+1x,y:=Q​\(θ,ϕ\)Q\_\{t\+1\}^\{x,y\}:=Q\(\\theta,\\phi\)for𝒟t\+1θ\\mathcal\{D\}^\{\\theta\}\_\{t\+1\}and𝒟t\+1ϕ\\mathcal\{D\}^\{\\phi\}\_\{t\+1\}, respectively, to explicitly emphasize this dependency\. Depending on whether the model updates occur synchronously or asynchronously \(Fig\.[1](https://arxiv.org/html/2605.29267#S2.F1)\(B1\)\), we consider the following update regimes:

1. 1\.Synchronous updates, where both modelsθt\\theta\_\{t\}andϕt\\phi\_\{t\}are updated simultaneously\. In this case,Pt\+1x,y:=P​\(θt,ϕt\)P\_\{t\+1\}^\{x,y\}:=P\(\\theta\_\{t\},\\phi\_\{t\}\)andQt\+1x,y:=Q​\(θt,ϕt\)Q\_\{t\+1\}^\{x,y\}:=Q\(\\theta\_\{t\},\\phi\_\{t\}\)\.
2. 2\.Asynchronous updates, where the update of one model \(e\.g\.,θt\+1\\theta\_\{t\+1\}\) occurs before the other \(e\.g\.,ϕt\+1\\phi\_\{t\+1\}\)\. In this setting, the self\-consuming synthetic data of the later\-updated model \(e\.g\.,𝒮t\+1ϕ\\mathcal\{S\}\_\{t\+1\}^\{\\phi\}\) includes samples generated by the already\-updated model \(e\.g\.,θt\+1\\theta\_\{t\+1\}\)\. As a result,Pt\+1x,y:=P​\(θt,ϕt\)P\_\{t\+1\}^\{x,y\}:=P\(\\theta\_\{t\},\\phi\_\{t\}\)andQt\+1x,y:=Q​\(θt\+1,ϕt\)Q\_\{t\+1\}^\{x,y\}:=Q\(\\theta\_\{t\+1\},\\phi\_\{t\}\)\.

The detailed sampling process is presented in Algorithm[1](https://arxiv.org/html/2605.29267#alg1)\. Given the new training datasets, both models are updated to minimize the expected loss under certain loss functions\.

θt\+1=arg⁡min𝜃​𝔼\(x,y\)∼Pt\+1x,y​\[ℓθ​\(x,y\)\];\\displaystyle\\theta\_\{t\+1\}=\\underset\{\\theta\}\{\\arg\\min\}\\ \\mathbb\{E\}\_\{\(x,y\)\\sim P\_\{t\+1\}^\{x,y\}\}\[\\ell\_\{\\theta\}\(x,y\)\];ϕt\+1=arg⁡minϕ​𝔼\(x,y\)∼Qt\+1x,y​\[ℓϕ​\(x,y\)\]\.\\displaystyle\\phi\_\{t\+1\}=\\underset\{\\phi\}\{\\arg\\min\}\\ \\mathbb\{E\}\_\{\(x,y\)\\sim Q\_\{t\+1\}^\{x,y\}\}\[\\ell\_\{\\phi\}\(x,y\)\]\.\(4\)
Objectives\.Recent studies have analyzed the evolution of self\-consuming generative models and the role of human curation in steering their behavior\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6); Wei and Zhang,[2025](https://arxiv.org/html/2605.29267#bib.bib10); Zhaoet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib15)\)\. However, prior work primarily focuses on a single, isolated model\. In multi\-model settings, it remains unclear how such an ecosystem evolves and how human curation at one model influences the overall system and its alignment in the long run\.

In this paper, we address these questions\. We first analyze the evolution of the self\-consuming multi\-model ecosystem introduced in this section, identifying conditions for the convergence and stability of the system \(Section[3](https://arxiv.org/html/2605.29267#S3)\)\. We then examine the impact of multi\-model interactions and human curation on convergence, with a particular focus on preference alignment \(Section[4](https://arxiv.org/html/2605.29267#S4)\)\.

## 3Evolution of Multi\-model Ecosystem

Next, we investigate how self\-consuming iterative training and cross\-model interactions drive the evolution of the multi\-model ecosystem\. Our goal is to characterize the conditions under which this ecosystem stabilizes and converges\. We first formalize the notion of stability for an evolving system\.

###### Definition 3\.1\(Stability\)\.

The multi\-model ecosystem reaches a stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)if the following holds\.

θ∗\\displaystyle\\theta^\{\*\}=arg⁡min𝜃​𝔼\(x,y\)∼P​\(θ∗,ϕ∗\)​\[ℓθ​\(x,y\)\];\\displaystyle=\\underset\{\\theta\}\{\\arg\\min\}\\ \\mathbb\{E\}\_\{\(x,y\)\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\[\\ell\_\{\\theta\}\(x,y\)\];ϕ∗\\displaystyle\\phi^\{\*\}=arg⁡minϕ​𝔼\(x,y\)∼Q​\(θ∗,ϕ∗\)​\[ℓϕ​\(x,y\)\]\.\\displaystyle=\\underset\{\\phi\}\{\\arg\\min\}\\ \\mathbb\{E\}\_\{\(x,y\)\\sim Q\(\\theta^\{\*\},\\phi^\{\*\}\)\}\[\\ell\_\{\\phi\}\(x,y\)\]\.\(5\)

At\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\), both models and the self\-consuming synthetic data they induce no longer change\. Next, we examine the evolution of\(θt,ϕt\)\(\\theta\_\{t\},\\phi\_\{t\}\)and identify conditions for the existence of a stable point and for convergence of the system\.

###### Assumption 3\.2\(Strong convexity\)\.

Loss functionℓθ\\ell\_\{\\theta\}isγθ\\gamma\_\{\\theta\}\-strongly convex inθ\\theta: for anyθ,θ′\\theta,\\theta^\{\\prime\}, we have

ℓθ​\(θ\)≥ℓθ​\(θ′\)\+∇θℓθ​\(θ′\)T​\(θ−θ′\)\+γθ2​‖θ−θ′‖2\\ell\_\{\\theta\}\(\\theta\)\\geq\\ell\_\{\\theta\}\(\\theta^\{\\prime\}\)\+\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\\prime\}\)^\{T\}\(\\theta\-\\theta^\{\\prime\}\)\+\\frac\{\\gamma\_\{\\theta\}\}\{2\}\|\|\\theta\-\\theta^\{\\prime\}\|\|^\{2\}\(6\)Similarly,ℓϕ\\ell\_\{\\phi\}isγϕ\\gamma\_\{\\phi\}\-strongly convex inϕ\\phi\.

###### Assumption 3\.3\(Smoothness\)\.

ℓθ​\(x,y\)\\ell\_\{\\theta\}\(x,y\)isLθL\_\{\\theta\}\-smooth in\(x,y\)\(x,y\)andθ\\theta: for any\(x,y\),\(x′,y′\),θ,θ′\(x,y\),\(x^\{\\prime\},y^\{\\prime\}\),\\theta,\\theta^\{\\prime\}, we have

‖∇θℓθ​\(x,y\)−∇θℓθ​\(x′,y′\)‖\\displaystyle\|\|\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(x,y\)\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(x^\{\\prime\},y^\{\\prime\}\)\|\|≤Lθ​‖\(x,y\)−\(x′,y′\)‖,\\displaystyle\\leq L\_\{\\theta\}\|\|\(x,y\)\-\(x^\{\\prime\},y^\{\\prime\}\)\|\|,\(7\)‖∇θℓθ​\(θ\)−∇θℓθ​\(θ′\)‖\\displaystyle\|\|\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta\)\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\\prime\}\)\|\|≤Lθ​‖θ−θ′‖\.\\displaystyle\\leq L\_\{\\theta\}\|\|\\theta\-\\theta^\{\\prime\}\|\|\.Similarly,ℓϕ​\(x,y\)\\ell\_\{\\phi\}\(x,y\)isLϕL\_\{\\phi\}\-smooth in\(x,y\)\(x,y\)andϕ\\phi\.

Note that Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)and[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)are standard and widely used in the literature on analyzing self\-consuming models\(Bertrandet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib9); Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\)and performative prediction\(Perdomoet al\.,[2020](https://arxiv.org/html/2605.29267#bib.bib29)\)\. These assumptions are sufficient to derive theorems, and empirical results in Section[5](https://arxiv.org/html/2605.29267#S5)suggest that the findings continue to hold even when these assumptions are relaxed or violated\.

###### Proposition 3\.4\(Convergence of multi\-model system\)\.

Suppose Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)and[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)hold, and data spaces𝒳\\mathcal\{X\},𝒴\\mathcal\{Y\}are bounded\. If both models are Lipschitz in their inputs and parametersθ\\theta,ϕ\\phi, and reward functions are Lipschitz in inputs, then there existsτ∈\(0,1\)\\tau\\in\(0,1\)such that if the fraction of real data in each round of model training is sufficiently large, i\.e\.,min⁡\(λℛθ,λℛϕ\)\>τ\\min\\big\(\\lambda^\{\\theta\}\_\{\\mathcal\{R\}\},\\lambda^\{\\phi\}\_\{\\mathcal\{R\}\}\\big\)\>\\tau, then a unique stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)exists\. Moreover, the iterative training loop \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\) will drive\(θt,ϕt\)\(\\theta\_\{t\},\\phi\_\{t\}\)to converge to\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\), and the training data distributions will also converge, i\.e\.,limt→∞Ptx,y=P​\(θ∗,ϕ∗\)\\lim\_\{t\\to\\infty\}P\_\{t\}^\{x,y\}=P\(\\theta^\{\*\},\\phi^\{\*\}\),limt→∞Qtx,y=Q​\(θ∗,ϕ∗\)\\lim\_\{t\\to\\infty\}Q\_\{t\}^\{x,y\}=Q\(\\theta^\{\*\},\\phi^\{\*\}\)\.

The convergence of a single, isolated self\-consuming model has been established byBertrandet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib9)\)\. Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4)implies that cross\-model interactions will not disrupt such convergence, provided that a sufficient amount of real data is included in each round of model updates\. The precise quantity ofτ\\tauis given in Appendix[F\.8](https://arxiv.org/html/2605.29267#A6.SS8)\. Indeed, the fractions of real data,λℛθ\\lambda^\{\\theta\}\_\{\\mathcal\{R\}\}andλℛϕ\\lambda^\{\\phi\}\_\{\\mathcal\{R\}\}, directly affect a property of this dynamic system known asdistribution sensitivity\. Rather than imposing a restriction onλℛθ\\lambda^\{\\theta\}\_\{\\mathcal\{R\}\}andλℛϕ\\lambda^\{\\phi\}\_\{\\mathcal\{R\}\}, we can instead formulate an alternative condition in terms of distribution sensitivity that also guarantees the stability and convergence\.

###### Assumption 3\.5\(Distribution sensitivity\)\.

The distributionP​\(θ,ϕ\)P\(\\theta,\\phi\)isεθ\\varepsilon\_\{\\theta\}\-sensitive: for any\(θ,ϕ\)\(\\theta,\\phi\),\(θ′,ϕ′\)\(\\theta^\{\\prime\},\\phi^\{\\prime\}\), we have

W​\(P​\(θ,ϕ\),P​\(θ′,ϕ′\)\)\\displaystyle W\\big\(P\(\\theta,\\phi\),P\(\\theta^\{\\prime\},\\phi^\{\\prime\}\)\\big\)≤εθ​‖\(θ,ϕ\)−\(θ′,ϕ′\)‖\.\\displaystyle\\leq\\varepsilon\_\{\\theta\}\|\|\(\\theta,\\phi\)\-\(\\theta^\{\\prime\},\\phi^\{\\prime\}\)\|\|\.whereW​\(⋅,⋅\)W\(\\cdot,\\cdot\)is the Wasserstein distance\. Similarly,Q​\(θ,β\)Q\(\\theta,\\beta\)isεϕ\\varepsilon\_\{\\phi\}\-sensitive\.

###### Theorem 3\.6\(Convergence of multi\-model system\)\.

Define

κ≜max⁡\(γθ​Lϕ​εϕ\+2​γϕ​Lθ​εθγϕ​\(γθ−Lθ​εθ\),γϕ​Lθ​εθ\+2​γθ​Lϕ​εϕγθ​\(γϕ−Lϕ​εϕ\),Lθ​εθγθ\+Lϕ​εϕγϕ\)\.\\displaystyle\\kappa\\triangleq\\max\\left\(\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\+2\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\phi\}\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\+2\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)\},\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\right\)\.

Under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2),[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3), and[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5), ifκ<1\\kappa<1, then the stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)exists, and iterative training loop \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\) will drive\(θt,ϕt\)\(\\theta\_\{t\},\\phi\_\{t\}\)to converge to\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)at a linear rate:

‖\(θt,ϕt\)−\(θ∗,ϕ∗\)‖≤κt​‖\(θ0,ϕ0\)−\(θ∗,ϕ∗\)‖\.\|\|\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\|\|\\leq\\kappa^\{t\}\|\|\(\\theta\_\{0\},\\phi\_\{0\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\|\|\.

In Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6), we consider all possible update orderings ofθt\\theta\_\{t\}andϕt\\phi\_\{t\}in each round, including both synchronous and asynchronous updates\. The form ofκ\\kappa, taking the maximum over three terms, ensures the convergence regardless of the update scheme\. We can relate Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4)to Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6): larger amounts of real data \(higher fractionsλℛθ\\lambda^\{\\theta\}\_\{\\mathcal\{R\}\}andλℛϕ\\lambda^\{\\phi\}\_\{\\mathcal\{R\}\}\) lead to less sensitive distributions \(smallerεθ\\varepsilon\_\{\\theta\}andεϕ\\varepsilon\_\{\\phi\}\) and, consequently, a smallerκ\\kappa\.

## 4Impact of Human Curation

Given the convergence to stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\), we next examine the role of human curation in shaping the system’s behavior\. For a single, isolated model,Ferbachet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib6)\)showed that when synthetic data is curated by humans according to their preferences in each round, the model’s outputs can gradually align with human preferences, eventually converging to a distribution that maximizes user reward\. However, in a multi\-model ecosystem, it remains unclear how alignment is influenced by curation\. Through inter\-model interactions, the influence of human curation applied to one model can propagate to others in complex and nontrivial ways, potentially distorting or even reversing the intended effects of human curation on model alignment\.

In this section, we systematically examine this by analyzing how varying the fraction of human\-curated synthetic data for one model at each round affects the multi\-model system in the long run\. For modelsθt\\theta\_\{t\}andϕt\\phi\_\{t\}, we define thealignmentas

Jp​\(θt,ℰθ\)\\displaystyle J\_\{p\}\(\\theta\_\{t\},\\mathcal\{E\}\_\{\\theta\}\):=𝔼y∼ℰθ,x∼pθt​\(x\|y\)​\[rθ​\(x,y\)\],\\displaystyle=\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta\_\{t\}\}\(x\|y\)\}\[r\_\{\\theta\}\(x,y\)\],\(8\)Jq​\(ϕt,ℰϕ\)\\displaystyle J\_\{q\}\(\\phi\_\{t\},\\mathcal\{E\}\_\{\\phi\}\):=𝔼x∼ℰϕ,y∼qϕt​\(y\|x\)​\[rϕ​\(x,y\)\]\.\\displaystyle=\\mathbb\{E\}\_\{x\\sim\\mathcal\{E\}\_\{\\phi\},y\\sim q\_\{\\phi\_\{t\}\}\(y\|x\)\}\[r\_\{\\phi\}\(x,y\)\]\.where reward functionsrθ​\(x,y\)r\_\{\\theta\}\(x,y\),rϕ​\(x,y\)r\_\{\\phi\}\(x,y\)quantify how well the input\-output pair\(x,y\)\(x,y\)is aligned with users of modelsθ\\thetaandϕ\\phi, respectively\. Importantly,ℰθ\\mathcal\{E\}\_\{\\theta\}andℰϕ\\mathcal\{E\}\_\{\\phi\}are generic*evaluation*distributions \(e\.g\., test benchmarks or downstream task datasets\) that may or may not match the training distributionsPtyP^\{y\}\_\{t\}orQtxQ^\{x\}\_\{t\}; this reflects standard practice in which alignment is assessed on datasets that may differ from the training environment\. In our analysis, we are interested in the alignment of the models at the stable point:How varying fraction of human\-curated samplesλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\(orλℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\) of one model in each round could affect alignment of its own \(self\-influence\) and the other model \(cross\-influence\) at the stable point, i\.e\., bothJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)andJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)\.

Because the interactions betweenϕt\\phi\_\{t\}andθt\\theta\_\{t\}are symmetric, it suffices to analyze∂Jp​\(θ∗\)∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\(self\-influence\) and∂Jq​\(ϕ∗\)∂λℋθ\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\(cross\-influence\)\. Sinceλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}may vary for many reasons \(e\.g\., changes in the curated or real\-data pools\), we adopt a simplified but controlled setting in which variations inλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}arise solely from changes in the size of the curated datasetℋθ\\mathcal\{H\}^\{\\theta\}, while all other data sources remain fixed\. Other cases are discussed in Appendix[E](https://arxiv.org/html/2605.29267#A5)\. Our goal is to identify conditions under which increasing human\-curation leads to improved or degraded model alignment at the stable point\.

### 4\.1Quantifying Curation Impact onJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)andJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)

We first quantify self\-influence∂Jp​\(θ∗\)∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and cross\-influence∂Jq​\(ϕ∗\)∂λℋθ\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\. SinceJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)andJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)are expectations measuring the generalization performance of models, the fraction of human\-curated dataλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}affectsJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)\(resp\.Jp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)\) throughϕ∗\\phi^\{\*\}\(resp\.θ∗\\theta^\{\*\}\)\. By the chain rule, we have

∂Jp​\(θ∗\)∂λℋθ=∂Jp​\(θ∗\)∂θ∗⋅∂θ∗∂λℋθ;∂Jq​\(ϕ∗\)∂λℋθ=∂Jq​\(ϕ∗\)∂ϕ∗⋅∂ϕ∗∂λℋθ\.\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\theta^\{\*\}\}\\cdot\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\};~~\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\phi^\{\*\}\}\\cdot\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\.Note that both∂Jp​\(θ∗\)∂θ∗\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\theta^\{\*\}\}and∂Jq​\(ϕ∗\)∂ϕ∗\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\phi^\{\*\}\}depend only on the reward functions and the model structures\. The main challenge, however, is computing∂θ∗∂λℋθ\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and∂ϕ∗∂λℋθ\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}, which describe how the stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)responds to changes in the fraction of human\-curated data\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/x2.png)

Figure 2:When varying only the curation strength of modelθ\\theta\(λℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\), theθ\\thetamodel’s update direction \(∇θℓθ\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\) is mapped into the reward alignment space viaSpS\_\{p\}\(a\)\. The projected vector, together with∇θJp\\nabla\_\{\\theta\}J\_\{p\}, characterizes the self\-influence \(b\)\. In addition,∇θℓθ\\nabla\_\{\\theta\}\\ell\_\{\\theta\}is first mapped into the cross\-model parameter space viaCqC\_\{q\}\(c\), and then transformed bySqS\_\{q\}\(d\); combined with∇ϕJq\\nabla\_\{\\phi\}J\_\{q\}, it determines the cross\-influence \(e\)\.Define the gradients of objective functions in \([3\.1](https://arxiv.org/html/2605.29267#S3.Ex3)\) as

Fp​\(θ,ϕ,λℋθ\)≜𝔼\(x,y\)∼P​\(θ,ϕ\)​\[∇θℓθ​\(x,y\)\];\\displaystyle F\_\{p\}\(\\theta,\\phi,\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\\triangleq\\mathbb\{E\}\_\{\(x,y\)\\sim P\(\\theta,\\phi\)\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(x,y\)\];Fq​\(θ,ϕ,λℋθ\)≜𝔼\(x,y\)∼Q​\(θ,ϕ\)​\[∇ϕℓϕ​\(x,y\)\]\.\\displaystyle F\_\{q\}\(\\theta,\\phi,\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\\triangleq\\mathbb\{E\}\_\{\(x,y\)\\sim Q\(\\theta,\\phi\)\}\[\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(x,y\)\]\.Note thatθ\\thetaandϕ\\phialone cannot fully capture the influence ofλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}onFq,FqF\_\{q\},F\_\{q\}, becauseλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}also determines the mixing weights in the mixture distributionsQ​\(θ,ϕ\)Q\(\\theta,\\phi\)andP​\(θ,ϕ\)P\(\\theta,\\phi\)\.

For any value ofλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}, the resulting stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)minimizes the objectives in \([3\.1](https://arxiv.org/html/2605.29267#S3.Ex3)\)\. Thus, the following holds

∀λℋθ,Fp​\(θ∗,ϕ∗,λℋθ\)=Fq​\(θ∗,ϕ∗,λℋθ\)=0,\\forall\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},~~F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)=F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)=0,which yields the following proposition\.

###### Proposition 4\.1\(Characterization of∂θ∗∂λℋθ\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and∂ϕ∗∂λℋθ\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\)\.

∂θ∗∂λℋθ=−Sp​\(∂Fp∂λℋθ\+Cp​∂Fq∂λℋθ\)\\displaystyle\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-S\_\{p\}\\left\(\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+C\_\{p\}\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\)∂ϕ∗∂λℋθ=−Sq​\(Cq​∂Fp∂λℋθ\+∂Fq∂λℋθ\)\\displaystyle\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-S\_\{q\}\\left\(C\_\{q\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\)

Here,Sp,SqS\_\{p\},S\_\{q\}are matrices characterizing thelocal sensitivityof each model’s stable point to perturbations, after accounting for the coupled dependence on the other model\. The matricesCp,CqC\_\{p\},C\_\{q\}quantifycross\-model influence, capturing how changes in one model’s objective induce parameter shifts in the other model through the coupled training dynamics\. We formally define these matrices below\.

###### Definition 4\.2\(Sensitivity matrix\)\.

ForF∈\{Fp,Fq\}F\\in\\\{F\_\{p\},F\_\{q\}\\\}, let the Jacobian matrices of the gradient mappingsFFevaluated at the stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)be

∇θ𝑭:=∇θF​\(θ∗,ϕ∗,λℋθ\);∇ϕ𝑭:=∇ϕF​\(θ∗,ϕ∗,λℋθ\)\.\\nabla\_\{\\theta\}\\boldsymbol\{F\}:=\\nabla\_\{\\theta\}F\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\);~~\\nabla\_\{\\phi\}\\boldsymbol\{F\}:=\\nabla\_\{\\phi\}F\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\.Thelocal sensitivitymatrices, which capture how each model’s parameters respond to perturbations while accounting for cross\-model interactions, are defined as

Sp:=\(∇θ𝐅p−∇ϕ𝐅p​\(∇ϕ𝐅q\)−1​∇θ𝐅q\)−1,\\displaystyle S\_\{p\}:=\\left\(\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\-\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\(\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\)^\{\-1\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\right\)^\{\-1\},Sq:=\(∇ϕ𝐅q−∇θ𝐅q​\(∇θ𝐅p\)−1​∇ϕ𝐅p\)−1\.\\displaystyle S\_\{q\}:=\\left\(\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\-\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\(\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\)^\{\-1\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\right\)^\{\-1\}\.\(9\)

###### Definition 4\.3\(Cross\-model influence matrix\)\.

Define

Cp:=−∇ϕ𝐅p​\(∇ϕ𝐅q\)−1,Cq:=−∇θ𝐅q​\(∇θ𝐅p\)−1\.\\displaystyle C\_\{p\}:=\-\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\,\(\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\)^\{\-1\},~~C\_\{q\}:=\-\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\,\(\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\)^\{\-1\}\.which quantify how changes in one model’s parameters propagate to the other model\.

If there is no cross\-model interaction \(i\.e\., each model’s input does not include data generated by the other model\), the system reduces to a single\-model scenario andλθϕ=λϕθ=0\\lambda\_\{\\theta\}^\{\\phi\}=\\lambda\_\{\\phi\}^\{\\theta\}=0\. In this case, the matrices simplify toSq=\(∇ϕ𝐅q\)−1S\_\{q\}=\(\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\)^\{\-1\},Sp=\(∇θ𝐅p\)−1S\_\{p\}=\(\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\)^\{\-1\}, andCq=Cp=𝟎C\_\{q\}=C\_\{p\}=\\mathbf\{0\}\.

###### Proposition 4\.4\.

Under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6),∇θ𝐅p\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\},∇ϕ𝐅q\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\},SpS\_\{p\}andSqS\_\{q\}are all invertible\.

Combining the above, we can quantify the influence of human\-curated data on the alignment of the multi\-model system at the stable point, as shown in Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)\.

###### Theorem 4\.5\(Self/cross\-influence quantification\)\.

Under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6), the self\-consuming multi\-model system converges\. LetPℋP\_\{\\mathcal\{H\}\}be the distribution of human\-curated dataℋtθ\\mathcal\{H\}^\{\\theta\}\_\{t\}after convergence\.

∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=11−λℋθ​⟨∇θJp​\(θ∗\),Sp​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩,\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}\\,\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle,∂Jq​\(ϕ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=11−λℋθ​⟨∇ϕJq​\(ϕ∗\),Sq​Cq​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\\,S\_\{q\}C\_\{q\}\\,\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle\.whereSpS\_\{p\},SqS\_\{q\}are the sensitivity matrices,CqC\_\{q\}is the cross\-model influence matrix, and⟨⋅,⋅⟩\\langle\\cdot,\\cdot\\rangledenotes the inner product between two vectors\.

In Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5),𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]represents the direction in whichθ\\thetawould be updated if updated using only human\-curated data\.∇θJp​\(θ∗\)\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)is the direction in parameter spaceΘ\\Thetathat most increases the alignment\. We can measure the alignment of these two directions using cosine similarity\.

ρp≜cos⁡\(∇θJp​\(θ∗\),𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]\)∈\[−1,1\],\\rho\_\{p\}\\triangleq\\cos\\left\(\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\right\)\\in\[\-1,1\],\(10\)Intuitively, if these two directions are aligned, i\.e\.,ρp\>0\\rho\_\{p\}\>0, then the update induced by human\-curated data coincides with the direction that improves the alignment metric\. One might therefore expect that increasing the fraction of human\-curated data would always enhance alignment\. This intuition is indeed correct in the single\-model setting when curation directly optimizes the alignment objective, in which caseρp≈1\\rho\_\{p\}\\approx 1; see Corollary[E\.1](https://arxiv.org/html/2605.29267#A5.Thmtheorem1)in Section[E\.1](https://arxiv.org/html/2605.29267#A5.SS1)\.

However, in a multi\-model ecosystem, this intuition does not necessarily hold, for two distinct reasons\. First, models can interact whenever they are trained on data of the same modality, even if the curation data distribution does not overlap with the evaluation distribution that definesJpJ\_\{p\}andJqJ\_\{q\}\. We refer to this phenomenon aspreference domain mismatch\(PDM\)\. Under PDM, model interactions can substantially alter parameters without inducing corresponding changes inJpJ\_\{p\}andJqJ\_\{q\}; see Section[D\.2](https://arxiv.org/html/2605.29267#A4.SS2)for further discussion\.

Second, even without PDM, model interactions can introduce conflicting curation signals:ℋtθ\\mathcal\{H\}\_\{t\}^\{\\theta\}may include cross\-model data whose induced update direction is misaligned with the alignment objective, resulting inρp<0\\rho\_\{p\}<0\. Moreover, Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)shows that cross\-model interactions and self\-consuming dynamics can further distort the impact of human curation through the sensitivity matrixSpS\_\{p\}\. In Section[4\.2](https://arxiv.org/html/2605.29267#S4.SS2), we illustrate that even whenρp\>0\\rho\_\{p\}\>0, human curation can sometimes reduce alignment across the ecosystem\.

### 4\.2Distortion Effect of Sensitivity Matrix

Next, we illustrate howSpS\_\{p\}can distort the impact of human\-curated data\. In particular, we provide an example showing that even whenρp\>0\\rho\_\{p\}\>0, this distortion can result in∂Jp​\(θ∗\)∂λℋθ<0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}<0, meaning that human curation intended to improve model alignment can, counterintuitively, reduce it\.

###### Example 4\.6\(ρp\>0​⟹\\rho\_\{p\}\>0\\not\\\!\\Longrightarrowpositive effect of human curation\)\.

Consider a simplified text\-image system\. For simplicity, we assume that each model’s output representation coincides with its parameters:

- •Text modelgenerates captionsθ^​\(ϕ,λℋθ\)=W1​ϕ\+a​λℋθ\\hat\{\\theta\}\(\\phi,\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)=W\_\{1\}\\phi\+a\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}describing input imagesϕ\\phi, whereW1W\_\{1\}encodes how the text representation is encouraged to align with the image representation \(e\.g\., to produce captions that are easy for the image model to pass safety filters\), andaaencodes the direction in representation space favored by human\-curated text whenλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}increases\. We assume the mixture distribution of the text model isP=𝒩​\(θ^​\(β,λℋθ\),σθ2​I\)P=\\mathcal\{N\}\(\\hat\{\\theta\}\(\\beta,\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\),\\sigma\_\{\\theta\}^\{2\}I\)with quadratic lossℓθ​\(z\)=12​∥θ−z∥2\\ell\_\{\\theta\}\(z\)=\\frac\{1\}\{2\}\\lVert\\theta\-z\\rVert^\{2\},z∼Pz\\sim P\.
- •Image modelgenerates imagesϕ^​\(θ\)=W2​θ\\hat\{\\phi\}\(\\theta\)=W\_\{2\}\\thetabased on input captionsθ\\theta, whereW2W\_\{2\}encodes the alignment between the image and text representations \(e\.g\., CLIP\-style alignment\)\. The mixture data distribution isQ=𝒩​\(ϕ^​\(θ\),σϕ2​I\)Q=\\mathcal\{N\}\(\\hat\{\\phi\}\(\\theta\),\\sigma\_\{\\phi\}^\{2\}I\)with lossℓϕ​\(z\)=12​∥ϕ−z∥2\\ell\_\{\\phi\}\(z\)=\\frac\{1\}\{2\}\\lVert\\phi\-z\\rVert^\{2\},z∼Qz\\sim Q\.

For this system, the gradients and Jacobians are

Fp=θ−W1​ϕ−a​λℋθ;Fq=ϕ−W2​θ\\displaystyle\{F\}\_\{p\}=\\theta\-W\_\{1\}\\phi\-a\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\};~~\{F\}\_\{q\}=\\phi\-W\_\{2\}\\theta∇θ𝑭p=I,∇ϕ𝑭p=−W1,∇θ𝑭q=−W2,∇ϕ𝑭q=I\\displaystyle\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{p\}=I,\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{p\}=\-W\_\{1\},\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{q\}=\-W\_\{2\},\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{q\}=IThus, the sensitivity matrixSp=\(I−W1​W2\)−1S\_\{p\}=\(I\-W\_\{1\}W\_\{2\}\)^\{\-1\}\.

Consider a specific setting with parameters defined below:

W1=I,W2=17​\[−9 66 3\],Jp​\(θ\)=⟨θ,\[10\]⟩,a=\[1−1\],\\displaystyle W\_\{1\}=I,W\_\{2\}=\\frac\{1\}\{7\}\\begin\{bmatrix\}\-9\\ 6\\\\ \\ 6\\ \\ 3\\end\{bmatrix\},J\_\{p\}\(\\theta\)=\\langle\\theta,\\begin\{bmatrix\}1\\\\ 0\\end\{bmatrix\}\\rangle,a=\\begin\{bmatrix\}1\\\\ \-1\\end\{bmatrix\},Since𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]=a\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]=aand∇θJp​\(θ\)=\[1,0\]T,∀θ\\nabla\_\{\\theta\}J\_\{p\}\(\\theta\)=\[1,0\]^\{T\},\\forall\\theta,

ρp=cos⁡\(∇θJp​\(θ∗\),𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]\)=22\>0\.\\rho\_\{p\}=\\cos\\left\(\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\right\)=\\frac\{\\sqrt\{2\}\}\{2\}\>0\.However,

∂Jp​\(θ∗\)∂λℋθ∝⟨∇θJp​\(θ∗\),Sp​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩=−12<0\.\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\propto\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}\\,\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle=\-\\frac\{1\}\{2\}<0\.

This example illustrates that, under strong cross\-model interactions viaW1W\_\{1\}andW2W\_\{2\}, even when the human\-curated data points in a locally reward\-improving direction \(ρp\>0\\rho\_\{p\}\>0\), the dynamics of the coupled text\-image system can invert its effect due to distortion induced by the sensitivity matrix\.

### 4\.3Positive and Negative Impact of Human Curation

Sinceρp\>0\\rho\_\{p\}\>0does not necessarily imply a positive effect of human curation, a natural question is whether we can identify conditions, expressed in terms ofρp\\rho\_\{p\}, under which increasing the fraction of human\-curated data is guaranteed to improve alignment in a multi\-model system\.

Self\-influence\.We first examine conditions under which increasing the fraction of human\-curated samples for modelθ\\thetaimproves or degrades its own alignmentJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)\.

###### Corollary 4\.7\.

Denoteτp=γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\\tau\_\{p\}=\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}and letmpm\_\{p\}be the minimal eigenvalue ofSp\+SpT2\\frac\{S\_\{p\}\+S\_\{p\}^\{T\}\}\{2\}, we have∥Sp∥≤1τp\\lVert S\_\{p\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{p\}\}and if\|ρp\|\>11\+mp2​τp2\|\\rho\_\{p\}\|\>\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\}, thensign⁡\(ρp\)⋅∂Jp​\(θ∗\)∂λℋθ\>0\\operatorname\{sign\}\(\\rho\_\{p\}\)\\cdot\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0\.

∥Sp∥\\lVert S\_\{p\}\\rVertquantifies the extent to whichSpS\_\{p\}distorts directional relationships\. A larger∥Sp∥\\lVert S\_\{p\}\\rVertimplies that discrepancies across dimensions between∇θJp​\(θ∗\)\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)and𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]may be amplified after the mapping\. By Eq\. \([4\.2](https://arxiv.org/html/2605.29267#S4.Ex14)\),∥Sp∥\\lVert S\_\{p\}\\rVertdepends on the interaction settings and the fraction of data originating from other models \(λθϕ,λϕθ\\lambda\_\{\\theta\}^\{\\phi\},\\lambda\_\{\\phi\}^\{\\theta\}\)\. Consequently, increasing curation ratio is beneficial when curation update direction is sufficiently aligned with the reward\-improving direction \(largeρp\>0\\rho\_\{p\}\>0\), and the coupled dynamics do not distort these directions too strongly, which is captured bymp,τpm\_\{p\},\\tau\_\{p\}\. Corollary[4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7)combines these two factors and gives sufficient conditions, for example, ifρp\>0\\rho\_\{p\}\>0is large enough to overcome the distortion caused by coupling iterations \(ρp\>11\+mp2​τp2\\rho\_\{p\}\>\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\}\), thensign⁡\(ρp\)=1\\operatorname\{sign\}\(\\rho\_\{p\}\)=1and∂Jp​\(θ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0\.

[Corollary4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7)is further validated by Example[4\.6](https://arxiv.org/html/2605.29267#S4.Thmtheorem6), where\|ρp\|=22<11\+mp2/∥Sp∥2≈0\.997≤11\+mp2​τp2\|\\rho\_\{p\}\|=\\frac\{\\sqrt\{2\}\}\{2\}<\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}/\\lVert S\_\{p\}\\rVert^\{2\}\}\}\\approx 0\.997\\leq\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\}\.

Cross\-model influence\.Next, we examine how human curation applied to training modelθ\\thetacan propagate to another modelϕ\\phiand improve or degrade its alignmentJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)\.

Similar to self\-influence, we measure the alignment between∇ϕJp​\(ϕ∗\)\\nabla\_\{\\phi\}J\_\{p\}\(\\phi^\{\*\}\), the direction in spaceΦ\\Phithat would most increase the alignment metric, andCq​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]C\_\{q\}\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\], the direction by which human curation forθ\\thetaindirectly shifts the parametersϕ\\phi\. Note that we multiply𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]by cross\-model influence matrixCqC\_\{q\}to translate the effect of changes inθ\\thetainto the parameter spaceΦ\\Phi\.

ρq≜cos⁡\(∇ϕJq​\(ϕ∗\),Cq​𝔼Pℋ​\[−∇θℓθ\]\)∈\[−1,1\]\.\\rho\_\{q\}\\triangleq\\cos\\left\(\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),C\_\{q\}\\ \\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\right\)\\in\[\-1,1\]\.\(11\)Similar to[Corollary4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7), we can then identify conditions, expressed in terms ofρq\\rho\_\{q\}, under which human curation ofθ\\thetapositively or negatively affects the long\-term alignment of another modelϕ\\phi\. See[CorollaryE\.2](https://arxiv.org/html/2605.29267#A5.Thmtheorem2)in[SectionE\.2](https://arxiv.org/html/2605.29267#A5.SS2)\.

### 4\.4From Local Sensitivity to Global Deviation

Our analysis so far has focused on the local sensitivities∂Jq​\(ϕ∗\)∂λℋθ\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and∂Jp​\(θ∗\)∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}, which quantify how model alignment responds*locally*to changes in the human\-curation weight\. A natural next question is how the*final*alignment metricsJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)andJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)are affected at the fixed point\. The following theorem characterizes, relative to a setting without self\-consuming data \(i\.e\., when all training data is real andλℛθ=λℛϕ=1\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}=1\), how much deviation in alignment synthetic data can induce at the fixed point\. Denote byθ∗​\(λℛθ\)\\theta^\{\*\}\(\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\),ϕ∗​\(λℛϕ\)\\phi^\{\*\}\(\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\)the stable points of the system obtained when the real\-data weights areλℛθ\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}andλℛϕ\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}, respectively\.

###### Theorem 4\.8\.

Under the conditions of Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4)and Assumption[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5), for anyλ^ℛθ,λ^ℛϕ\>τ\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\},\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\phi\}\>\\tauas in Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4),

\|Jp​\(θ∗​\(λ^ℛθ\)\)−Jp​\(θ∗​\(1\)\)\|=𝒪​\(1−λ^ℛθ\),\\displaystyle\\left\|J\_\{p\}\\left\(\\theta^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\right\)\\right\)\-J\_\{p\}\\left\(\\theta^\{\*\}\\Big\(1\\Big\)\\right\)\\right\|=\\mathcal\{O\}\\Big\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\Big\),\|Jq​\(ϕ∗​\(λ^ℛϕ\)\)−Jq​\(ϕ∗​\(1\)\)\|=𝒪​\(1−λ^ℛϕ\)\.\\displaystyle\\left\|J\_\{q\}\\left\(\\phi^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\phi\}\\right\)\\right\)\-J\_\{q\}\\Big\(\\phi^\{\*\}\\Big\(1\\Big\)\\Big\)\\right\|=\\mathcal\{O\}\\Big\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\phi\}\\Big\)\.

Theorem[4\.8](https://arxiv.org/html/2605.29267#S4.Thmtheorem8)quantifies how much the final alignment scores can deviate from the ideal setting in which models are trained exclusively on real data\. It shows that as long as the real\-data weights remain above a stability thresholdτ\\tau, the deviation in alignment scales at most linearly with the amount of synthetic data\.

Discussion\.In[AppendixB](https://arxiv.org/html/2605.29267#A2), we discuss more about the limitations of our framework, experiments and future works\.

## 5Experiments

We consider a two\-model system initialized from base modelsθ0\\theta\_\{0\}andϕ0\\phi\_\{0\}\. Following the self\-consuming training paradigm described in Section[2](https://arxiv.org/html/2605.29267#S2), both models are updated iteratively using data that mix real samples with \(human\-curated\) synthetic data\. We conduct experiments on:

\(1\) Gaussian models: We extend Example[4\.6](https://arxiv.org/html/2605.29267#S4.Thmtheorem6)to validate the mechanism more detailed\. Letθ,ϕ∈ℝ24\\theta,\\phi\\in\\mathbb\{R\}^\{24\}, and the mixture distributionsP=\(1−λℋθ\)​𝒩​\(ϕ,σ2​I\)\+λℋθ​𝒩​\(ϕ\+a,σ2​I\)P=\(1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\\mathcal\{N\}\(\\phi,\\sigma^\{2\}I\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\\mathcal\{N\}\(\\phi\+a,\\sigma^\{2\}I\)andQ=𝒩​\(A​\(t\)​θ,σ2​I\)Q=\\mathcal\{N\}\(A\(t\)\\theta,\\sigma^\{2\}I\)whereA​\(t\)∈ℝ24×24A\(t\)\\in\\mathbb\{R\}^\{24\\times 24\}is a block\-diagonal matrix with 122×22\\times 2blocks\. Theii\-th blockAi​\(t\)=t​βi​RiA\_\{i\}\(t\)=t\\beta\_\{i\}R\_\{i\}wherettis the coupling scale,βi\\beta\_\{i\}controls the interaction magnitude of blockii, andRiR\_\{i\}is a 2\-dimensional orthogonal matrix introduces local geometric heterogeneity\. We set the alignmentsJp​\(θ\)=gpT​θ−ηp​∥θ∥22J\_\{p\}\(\\theta\)=g\_\{p\}^\{T\}\\theta\-\\frac\{\\eta\_\{p\}\\lVert\\theta\\rVert^\{2\}\}\{2\}andJq​\(ϕ\)=gqT​ϕ−ηq​∥ϕ∥22J\_\{q\}\(\\phi\)=g\_\{q\}^\{T\}\\phi\-\\frac\{\\eta\_\{q\}\\lVert\\phi\\rVert^\{2\}\}\{2\}, and lossesℓp​\(θ;z\)=12​∥θ−z∥2\\ell\_\{p\}\(\\theta;z\)=\\frac\{1\}\{2\}\\lVert\\theta\-z\\rVert^\{2\},ℓq​\(ϕ;z\)=12​∥ϕ−z∥2\\ell\_\{q\}\(\\phi;z\)=\\frac\{1\}\{2\}\\lVert\\phi\-z\\rVert^\{2\}\.a,gp,gq,Ri,βia,g\_\{p\},g\_\{q\},R\_\{i\},\\beta\_\{i\}are all pre\-set \(details in Section[D\.3](https://arxiv.org/html/2605.29267#A4.SS3)\)\. The system is constructed from orthogonal block matrixRiR\_\{i\}with matched blockwise curation and reward directionsa,gp,gqa,g\_\{p\},g\_\{q\}\. This yields a multi\-mode coupled system that remains analytically tractable, cleanly separates coupling magnitude from geometric distortion\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/figures/rebuttal_exp.png)

Figure 3:Gaussian experiment results\. \(A\):ρp,ρq\\rho\_\{p\},\\rho\_\{q\}in Eq\. \([10](https://arxiv.org/html/2605.29267#S4.E10)\)\-\([11](https://arxiv.org/html/2605.29267#S4.E11)\) for various coupling scalettwith fixedλℋθ=0\.4\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.4\. \(B\): The theoretical explicit∂Jp​\(θ∗\)∂λℋθ,∂Jq​\(ϕ∗\)∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\},\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and RHS in Theorem 4\.5 fort∈\[0\.05,1\]t\\in\[0\.05,1\]with fixedλℋθ=0\.4\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.4\. \(C\)\(D\): The theoretical explicit∂Jp​\(θ∗\)∂λℋθ,∂Jq​\(ϕ∗\)∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\},\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and their empirical values from finite samples \(of sizen=4/12/64n=4/12/64\) with 95% confidence intervals\. \(E\): The components of⟨∇θJp​\(θ∗\),𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangleand⟨∇θJp​\(θ∗\),Sp​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\ranglewithin specific dimension blocks\. \(F\): The components of⟨∇ϕJq​\(ϕ∗\),Cq​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),C\_\{q\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangleand⟨∇ϕJq​\(ϕ∗\),Sq​Cq​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),S\_\{q\}C\_\{q\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\ranglewithin specific dimension blocks\. \(G\)\(H\): ExplicitJp​\(θ∗\),Jq​\(ϕ∗\)J\_\{p\}\(\\theta^\{\*\}\),J\_\{q\}\(\\phi^\{\*\}\)and their finite\-sample empirical estimators witht∈\{0\.2,0\.9\}t\\in\\\{0\.2,0\.9\\\}for variousλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}with 95% confidence intervals\.\(2\) CIFAR\-10\(Krizhevskyet al\.,[2009](https://arxiv.org/html/2605.29267#bib.bib34)\): Bothθ\\thetaandϕ\\phiare implemented as class\-conditional diffusion models\(von Platenet al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib60)\)that generate images conditioned on class labels\. The base modelsθ0\\theta\_\{0\}andϕ0\\phi\_\{0\}are trained using 25k real samples\. Human curation is implemented via hue\-based reward functionsrθr\_\{\\theta\}andrϕr\_\{\\phi\}\(see Appendix[D\.1](https://arxiv.org/html/2605.29267#A4.SS1)for details\), which induce conflicting preferences: modelθ\\thetaprefers warm\-toned images, while modelϕ\\phiprefers cool\-toned images\.

\(3\) Qwen2\.5\-0\.5B\(Team,[2024](https://arxiv.org/html/2605.29267#bib.bib65)\): Modelθ\\thetais trained to summarize long articles, while modelϕ\\phiparaphrases relatively short texts\. Both models are initialized from Qwen2\.5\-0\.5B\-Instruct as base modelsθ0\\theta\_\{0\}andϕ0\\phi\_\{0\}, each equipped with a separate LoRA adapter\. The reward functionsrθr\_\{\\theta\}andrϕr\_\{\\phi\}are task\-specific measures that evaluate how well modelsθ\\thetaandϕ\\phiperform summarization and paraphrasing, respectively\.

### 5\.1Mechanism Validation

We validate the theory mechanism in Section[4](https://arxiv.org/html/2605.29267#S4)\. Based on our setting and Eq\. \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\), the two models updates areθt\+1=ϕt\+λℋθ​a\\theta\_\{t\+1\}=\\phi\_\{t\}\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}a,ϕt\+1=A​\(t\)​θt\\phi\_\{t\+1\}=A\(t\)\\theta\_\{t\}\. We set the iteration number to 100\. As shown in Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(A\)\(B\), whenλℋθ=0\.4\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.4is fixed,ρp,ρq\>0\\rho\_\{p\},\\rho\_\{q\}\>0for all coupling scalest∈\[0\.05,1\]t\\in\[0\.05,1\], while the reward gradient of modelϕ\\phidrops below 0 which aligns with the conclusion in Example[4\.6](https://arxiv.org/html/2605.29267#S4.Thmtheorem6)\. Moreover, the RHS of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)matches the derivative obtained from the closed\-form expressions ofJp,JqJ\_\{p\},J\_\{q\}\(Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(B\)\), confirming the theorem in this analyzable regime\. Furthermore, when the theory predicts a positive derivative \(fixλℋθ=0\.4\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.4,∂Jp​\(θ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0att=0\.2​or​0\.9t=0\.2\\ \\text\{or\}\\ 0\.9and∂Jq​\(ϕ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0att=0\.2t=0\.2in Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(B\)\), theJpJ\_\{p\}andJqJ\_\{q\}indeed increase under small curation perturbations inλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\(Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(G\)\(H\)\); fort=0\.9t=0\.9,∂Jq​\(ϕ∗\)∂λℋθ<0\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}<0\(Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(B\)\), andJqJ\_\{q\}decreases under small curation perturbations inλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\(Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(H\)\)\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/x3.png)

Figure 4:Left: The model parameter update ratiosΔt,j​\(φ\),φ∈\{θ,ϕ\},j∈\{2,5\}\\Delta\_\{t,j\}\(\\varphi\),\\ \\varphi\\in\\\{\\theta,\\phi\\\},j\\in\\\{2,5\\\}and model rewardsJp​\(θt\)J\_\{p\}\(\\theta\_\{t\}\),Jq​\(ϕt\)J\_\{q\}\(\\phi\_\{t\}\)forA​4A4withλℋθ=0\.6\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.6for modelsθ\\thetaandϕ\\phiat different iterations\. Middle: Expected model rewardsJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)andJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)under settingsA​1A1\-A​6A6after convergence\. Right: The proportions of cross\-model synthetic data for modelsθ\\thetaandϕ\\phiin settingsA​1A1\-A​6A6\.Next, we examine the self and cross\-influence in the system\. We focus on the dimensional contributions of the key inner products in Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)\. Based on the blockwise Gaussian settings, the overall self and cross influence can be decomposed into the sum of influences across different dimensions in the system \(details provided in Appendix[D\.3](https://arxiv.org/html/2605.29267#A4.SS3)\)\. Moreover,⟨∇θJp​\(θ∗\),𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangleand⟨∇ϕJq​\(ϕ∗\),Cq​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),C\_\{q\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\ranglemeasure whether the curation\-induced update direction is locally aligned with the reward\-improving direction, while⟨∇θJp​\(θ∗\),𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangleand⟨∇ϕJq​\(ϕ∗\),Sq​Cq​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),S\_\{q\}C\_\{q\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\ranglemeasure the actual self and cross\-influence\. As shown in Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(E\)\(F\),SpS\_\{p\}andSq​CqS\_\{q\}C\_\{q\}can amplify, attenuate, or even reverse self and cross influences in different dimensions, and the dominant dimensions before and after projection can differ substantially\. This provides a mechanism\-level explanation for whyρp\>0\\rho\_\{p\}\>0does not imply a positive curation effect\.

Finally, we focus on the statistical error between the finite\-sample estimates and the theoretical values\. Letn∈\[4,64\]n\\in\[4,64\]be the sample size used for updating models\. By comparing the closed\-formJp,JqJ\_\{p\},J\_\{q\}\(Fig\.[3](https://arxiv.org/html/2605.29267#S5.F3)\(G\)\(H\)\) and their derivatives of curation ratio \(Fig\.[3](https://arxiv.org/html/2605.29267#S5.F3)\(C\)\(D\)\) with finite\-sample estimates, the statistical error decreases rapidly asnnincreases\.

### 5\.2Convergence and The Impact of Human Curation

We examine system convergence and the effect of human curation on CIFAR\-10 using six settings \(A​1A1–A​6A6\)\. In all settings, each training round uses 8k samples withλℋθ=λℋϕ=0\.5\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}=0\.5andλℛθ=λℛϕ=0\.5\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}=0\.5\. To assess the impact of human curation, we add 2,000 curated samples for modelθ\\theta, increasingλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}to 0\.6 while keeping all other proportions fixed\. Six settings differ only in the fraction of cross\-model synthetic data\(λϕθ,λθϕ\)\(\\lambda\_\{\\phi\}^\{\\theta\},\\lambda\_\{\\theta\}^\{\\phi\}\): as shown in Figure[4](https://arxiv.org/html/2605.29267#S5.F4)\(right\), fromA​1A1toA​4A4this fraction increases for modelθ\\thetaand decreases forϕ\\phi, whileA​5A5andA​6A6serve as control cases\.

Convergence\.We measure convergence usingΔt,j​\(φ\)=∥φt−φt−j∥∥φt−j∥2\\Delta\_\{t,j\}\(\\varphi\)=\\frac\{\\lVert\\varphi\_\{t\}\-\\varphi\_\{t\-j\}\\rVert\}\{\\lVert\\varphi\_\{t\-j\}\\rVert\_\{2\}\},φ∈\{θ,ϕ\}\\varphi\\in\\\{\\theta,\\phi\\\}, which quantifies the relative change in model parameters\. As shown in Figure[4](https://arxiv.org/html/2605.29267#S5.F4)\(left\),Δt,j\\Delta\_\{t,j\}for both models drops below10−410^\{\-4\}and the rewards stabilize, indicating convergence\. We assume thatθ∗=θ54\\theta^\{\*\}=\\theta\_\{54\}andϕ∗=ϕ54\\phi^\{\*\}=\\phi\_\{54\}\. Additional results are in Appendix[D\.1\.3](https://arxiv.org/html/2605.29267#A4.SS1.SSS3)\.

Self\-influence\.Figure[4](https://arxiv.org/html/2605.29267#S5.F4)\(right\) shows that when interactions have limited impact on modelθ\\theta\(A​1A1–A​3A3andA​5A5, withλθϕ≤0\.3\\lambda\_\{\\theta\}^\{\\phi\}\\leq 0\.3\), strengthening curation forθ\\thetaincreasesJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)\. Asλθϕ\\lambda\_\{\\theta\}^\{\\phi\}grows, however, increasingλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}from 0\.5 to 0\.6 instead reducesJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\), indicating stronger distortion from model interactions\. Becauseθ\\thetaandϕ\\phihave conflicting preferences, stronger interaction causes the curation\-induced update𝔼Pℋ​\[−∇θℓθ\]\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]to deviate from∇θJp\\nabla\_\{\\theta\}J\_\{p\}, reducingρp\\rho\_\{p\}and potentially making it negative\. Consistent with Corollary[4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7), this leads to∂Jp​\(θ∗\)∂λℋθ<0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}<0inA​4A4\. Moreover, with the sameλθϕ\\lambda\_\{\\theta\}^\{\\phi\},A​2A2exhibits a larger improvement inJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)thanA​5A5becauseϕ\\phidepends more strongly onθ\\theta, which weakensϕ\\phi’s influence onθ\\theta\. The same mechanism explains the corresponding differences inJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)\.

Cross\-influence\.Due to the conflicting preferences, when modelϕ\\phi’s curation training data is sourced entirely from modelθ\\theta\(λϕθ=1\\lambda\_\{\\phi\}^\{\\theta\}=1\), increasing the amount of curation data tends to decreasesJqJ\_\{q\}\. Moreover, training on a large volume of preference\-conflicting samples \(λϕθ≥0\.4\\lambda\_\{\\phi\}^\{\\theta\}\\geq 0\.4\) can driveJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)close to or below 0, as observed in settingsA​1∼A​3A1\\sim A3andA​6A6in Figure[4](https://arxiv.org/html/2605.29267#S5.F4)\.

As the influence ofθ\\thetaonϕ\\phidecreases \(i\.e\., asλϕθ\\lambda\_\{\\phi\}^\{\\theta\}is reduced\), the projected curation\-induced updateCq​𝔼Pℋ​\[−∇θℓθ\]C\_\{q\}\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]becomes better aligned with modelϕ\\phi’s reward\-improving direction, leading to an increase inρq\\rho\_\{q\}\. By Corollary[E\.2](https://arxiv.org/html/2605.29267#A5.Thmtheorem2), whenρq\\rho\_\{q\}is sufficiently large,∂Jq​\(ϕ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0, which explains the improvement inJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)observed in settingA​4A4\. Finally, when the two models are strongly coupled \(A​6A6\), neither model produces high\-reward samples after convergence, rendering curation ineffective; consequently, rewards remain low and increasingλθϕ\\lambda\_\{\\theta\}^\{\\phi\}has little effect on eitherJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)orJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)\.

### 5\.3Preference Domain Mismatch \(PDM\) Experiments

![Refer to caption](https://arxiv.org/html/2605.29267v1/x4.png)

Figure 5:Left: modelsθ\\thetaandϕ\\phi’s rewards on their own evaluation datasets\. Right: Rewards on cross\-domain evaluation datasets\.Next, we use Qwen2\.5\-0\.5B to study preference domain mismatch \(PDM\)\. To isolate the effect of PDM, We consider an extreme setting withλθϕ=λϕθ=1\\lambda\_\{\\theta\}^\{\\phi\}=\\lambda\_\{\\phi\}^\{\\theta\}=1andλℛθ=λℛϕ=0\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}=0\. In this case, modelθ\\thetais trained exclusively on short texts but evaluated on long\-article summarization, while modelϕ\\phiis trained on long articles but evaluated on short\-text paraphrasing\. Consequently, cross\-model interactions induce training inputs that are mismatched with each model’s target evaluation inputs, resulting in PDM\.

We define cross\-domain rewardsJpc​r​o​s​s​\(θt\),Jqc​r​o​s​s​\(ϕt\)J\_\{p\}^\{cross\}\(\\theta\_\{t\}\),J\_\{q\}^\{cross\}\(\\phi\_\{t\}\)as the expected rewards when each model is evaluated on the other task’s evaluation inputs but scored using its own reward function\. As shown in Figure[5](https://arxiv.org/html/2605.29267#S5.F5), target rewardsJp​\(θt\)J\_\{p\}\(\\theta\_\{t\}\),Jq​\(ϕt\)J\_\{q\}\(\\phi\_\{t\}\)remain nearly unchanged, while the cross\-domain rewards improve\. This pattern indicates that cross\-only updates adapt each model to the other domain’s input distribution, but this adaptation transfers weakly to the target evaluation domain\. As a result, although modelθ\\thetatends to shorten inputs andϕ\\phitends to preserve input length, strong PDM obscures the effect of model coupling when preference alignment is measured on the target evaluation sets\.

## Impact Statement

This paper presents work whose goal is to advance the field of Machine Learning\. The contributions are theoretical in nature and focus on analyzing the dynamics of self\-consuming multi\-model iterative retraining and how the human curation influences the multi\-model ecosystem\. While Machine Learning systems may have broad societal impacts, we do not identify any specific ethical issues or foreseeable societal consequences that require separate discussion for this work\.

## Acknowledgements

This work was funded in part by the National Science Foundation under award number IIS2202699, IIS\-2416895, IIS\-2301599, CMMI2301601, and DMS\-2529302\.

## References

- S\. Alemohammad, J\. Casco\-Rodriguez, L\. Luzi, A\. I\. Humayun, H\. Babaei, D\. LeJeune, A\. Siahkoohi, and R\. Baraniuk \(2023\)Self\-consuming generative models go mad\.InThe Twelfth International Conference on Learning Representations,Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p3.1)\.
- S\. Basu, P\. Pope, and S\. Feizi \(2021\)Influence functions in deep learning are fragile\.InInternational Conference on Learning Representations \(ICLR\),Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p5.2),[§D\.3](https://arxiv.org/html/2605.29267#A4.SS3.p1.1)\.
- Q\. Bertrand, J\. Bose, A\. Duplessis, M\. Jiralerspong, and G\. Gidel \(2024\)On the stability of iterative retraining of generative models on their own data\.InThe Twelfth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=JORAfH2xFd)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p2.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p3.1),[§1](https://arxiv.org/html/2605.29267#S1.p2.1),[Figure 1](https://arxiv.org/html/2605.29267#S2.F1),[Figure 1](https://arxiv.org/html/2605.29267#S2.F1.13.13.6),[§3](https://arxiv.org/html/2605.29267#S3.p3.1),[§3](https://arxiv.org/html/2605.29267#S3.p4.5)\.
- F\. Bianchi, M\. Suzgun, G\. Attanasio, P\. Röttger, D\. Jurafsky, T\. Hashimoto, and J\. Y\. Zou \(2024\)Safety\-tuned llamas: lessons from improving the safety of large language models that follow instructions\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 34196–34216\.Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p6.1)\.
- R\. A\. Bradley and M\. E\. Terry \(1952\)Rank analysis of incomplete block designs: i\. the method of paired comparisons\.Biometrika39\(3/4\),pp\. 324–345\.Cited by:[item 3](https://arxiv.org/html/2605.29267#S2.I1.i3.p1.10)\.
- G\. Brown, S\. Hod, and I\. Kalemaj \(2022\)Performative prediction in a stateful world\.InInternational conference on artificial intelligence and statistics,pp\. 6045–6061\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1)\.
- Z\. Cai, Y\. Wang, Y\. Liu, and X\. Zhang \(2026\)Stabilizing self\-consuming diffusion models with latent space filtering\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 19844–19852\.Cited by:[§1](https://arxiv.org/html/2605.29267#S1.p2.1)\.
- T\. Chen, Y\. Hirota, M\. Otani, N\. Garcia, and Y\. Nakashima \(2024\)Would deep generative models amplify bias in future models?\.InProceedings of the IEEE/CVF conference on computer vision and pattern recognition,pp\. 10833–10843\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- P\. F\. Christiano, J\. Leike, T\. Brown, M\. Martic, S\. Legg, and D\. Amodei \(2017\)Deep reinforcement learning from human preferences\.Advances in neural information processing systems30\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1)\.
- J\. Cutler, M\. Diaz, and D\. Drusvyatskiy \(2024\)Stochastic approximation with decision\-dependent distributions: asymptotic normality and optimality\.Journal of Machine Learning Research25\(90\),pp\. 1–49\.Cited by:[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p1.1)\.
- J\. Dai, X\. Pan, R\. Sun, J\. Ji, X\. Xu, M\. Liu, Y\. Wang, and Y\. Yang \(2024\)Safe rlhf: safe reinforcement learning from human feedback\.InInternational Conference on Learning Representations,Vol\.2024,pp\. 50750–50777\.Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p6.1)\.
- E\. Dohmatob, Y\. Feng, A\. Subramonian, and J\. Kempe \(2025\)Strong model collapse\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=et5l9qPUhm)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- Y\. Feng, E\. Dohmatob, P\. Yang, F\. Charton, and J\. Kempe \(2025\)Beyond model collapse: scaling up with synthesized data requires verification\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=MQXrTMonT1)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1)\.
- D\. Ferbach, Q\. Bertrand, A\. J\. Bose, and G\. Gidel \(2024\)Self\-consuming generative models with curated data provably optimize human preferences\.InAdvances in Neural Information Processing Systems,Vol\.37,pp\. 102531–102567\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p2.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p3.1),[§D\.1\.3](https://arxiv.org/html/2605.29267#A4.SS1.SSS3.p2.6),[§1](https://arxiv.org/html/2605.29267#S1.p2.1),[§1](https://arxiv.org/html/2605.29267#S1.p5.1),[Figure 1](https://arxiv.org/html/2605.29267#S2.F1),[Figure 1](https://arxiv.org/html/2605.29267#S2.F1.13.13.6),[§2](https://arxiv.org/html/2605.29267#S2.p4.1),[§3](https://arxiv.org/html/2605.29267#S3.p3.1),[§4](https://arxiv.org/html/2605.29267#S4.p1.1)\.
- S\. Fu, Y\. Wang, Y\. Chen, X\. Tian, and D\. Tao \(2025\)A theoretical perspective: how to prevent model collapse in self\-consuming training loops\.InThe Thirteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=WttfQGwpES)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- S\. Fu, S\. Zhang, Y\. Wang, X\. Tian, and D\. Tao \(2024\)Towards theoretical understandings of self\-consuming generative models\.InProceedings of the 41st International Conference on Machine Learning,Proceedings of Machine Learning Research, Vol\.235,pp\. 14228–14255\.External Links:[Link](https://proceedings.mlr.press/v235/fu24i.html)Cited by:[§1](https://arxiv.org/html/2605.29267#S1.p2.1)\.
- W\. Gao and M\. Li \(2025\)Convergence dynamics and stabilization strategies of co\-evolving generative models\.External Links:2503\.08117,[Link](https://arxiv.org/abs/2503.08117)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p2.1),[§1](https://arxiv.org/html/2605.29267#S1.p4.1)\.
- M\. Gerstgrasser, R\. Schaeffer, A\. Dey, R\. Rafailov, T\. Korbak, H\. Sleight, R\. Agrawal, J\. Hughes, D\. B\. Pai, A\. Gromov, D\. Roberts, D\. Yang, D\. L\. Donoho, and S\. Koyejo \(2024\)Is model collapse inevitable? breaking the curse of recursion by accumulating real and synthetic data\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=5B2K4LRgmz)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- M\. Hardt, B\. Recht, and Y\. Singer \(2016\)Train faster, generalize better: stability of stochastic gradient descent\.InInternational conference on machine learning,pp\. 1225–1234\.Cited by:[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p1.1)\.
- J\. Ho, A\. Jain, and P\. Abbeel \(2020\)Denoising diffusion probabilistic models\.Advances in neural information processing systems33,pp\. 6840–6851\.Cited by:[§D\.1\.1](https://arxiv.org/html/2605.29267#A4.SS1.SSS1.p1.11)\.
- E\. J\. Hu, Y\. Shen, P\. Wallis, Z\. Allen\-Zhu, Y\. Li, S\. Wang, L\. Wang, W\. Chen,et al\.\(2022\)Lora: low\-rank adaptation of large language models\.\.ICLR1\(2\),pp\. 3\.Cited by:[§D\.2](https://arxiv.org/html/2605.29267#A4.SS2.p5.3)\.
- Z\. Hu, M\. Rostami, and J\. Thomason \(2025\)Multimodal synthetic data finetuning and model collapse: insights from vlms and diffusion models\.InProceedings of the 27th International Conference on Multimodal Interaction,ICMI ’25,New York, NY, USA,pp\. 588–599\.External Links:ISBN 9798400714993Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p4.1)\.
- K\. Jin, T\. Xie, Y\. Liu, and X\. Zhang \(2026\)Addressing polarization and unfairness in performative prediction\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.40,pp\. 22408–22416\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1)\.
- K\. Jin, T\. Yin, Z\. Chen, Z\. Sun, X\. Zhang, Y\. Liu, and M\. Liu \(2024\)Performative federated learning: a solution to model\-dependent and heterogeneous distribution shifts\.InProceedings of the AAAI Conference on Artificial Intelligence,Vol\.38,pp\. 12938–12946\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1)\.
- H\. Kim, G\. Papamakarios, and A\. Mnih \(2021\)The lipschitz constant of self\-attention\.InInternational Conference on Machine Learning,pp\. 5562–5571\.Cited by:[Remark F\.9](https://arxiv.org/html/2605.29267#A6.Thmtheorem9.p1.1)\.
- Y\. Kinoshita and H\. Kiya \(2020\)Hue\-correction scheme considering ciede2000 for color\-image enhancement including deep\-learning\-based algorithms\.APSIPA Transactions on Signal and Information Processing9,pp\. e19\.Cited by:[§D\.1\.2](https://arxiv.org/html/2605.29267#A4.SS1.SSS2.p2.1)\.
- A\. Krizhevsky, G\. Hinton,et al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[§5](https://arxiv.org/html/2605.29267#S5.p3.8)\.
- Q\. Li and H\. Wai \(2024\)Stochastic optimization schemes for performative prediction with nonconvex loss\.Advances in Neural Information Processing Systems37,pp\. 8673–8697\.Cited by:[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p2.1)\.
- Q\. Li, C\. Yau, and H\. Wai \(2022\)Multi\-agent performative prediction with greedy deployment and consensus seeking agents\.Advances in Neural Information Processing Systems35,pp\. 38449–38460\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1)\.
- X\. Li, J\. Luo, Z\. Zheng, H\. Wang, L\. Luo, L\. Wen, L\. Wu, and S\. Xu \(2025\)On the performance analysis of momentum method: a frequency domain perspective\.InInternational Conference on Learning Representations,Vol\.2025,pp\. 47713–47734\.Cited by:[§D\.3](https://arxiv.org/html/2605.29267#A4.SS3.p1.1)\.
- R\. D\. Luceet al\.\(1959\)Individual choice behavior\.Vol\.4,Wiley New York\.Cited by:[item 3](https://arxiv.org/html/2605.29267#S2.I1.i3.p1.10)\.
- L\. Meunier, B\. J\. Delattre, A\. Araujo, and A\. Allauzen \(2022\)A dynamical system perspective for lipschitz neural networks\.InInternational Conference on Machine Learning,pp\. 15484–15500\.Cited by:[Remark F\.9](https://arxiv.org/html/2605.29267#A6.Thmtheorem9.p1.1)\.
- M\. Mofakhami, I\. Mitliagkas, and G\. Gidel \(2023\)Performative prediction with neural networks\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 11079–11093\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p2.1)\.
- S\. Narayan, S\. B\. Cohen, and M\. Lapata \(2018\)Don’t give me the details, just the summary\! topic\-aware convolutional neural networks for extreme summarization\.InProceedings of the 2018 Conference on Empirical Methods in Natural Language Processing,Brussels, Belgium,pp\. 1797–1807\.External Links:[Link](https://aclanthology.org/D18-1206/),[Document](https://dx.doi.org/10.18653/v1/D18-1206)Cited by:[§D\.2](https://arxiv.org/html/2605.29267#A4.SS2.p6.11)\.
- L\. Ouyang, J\. Wu, X\. Jiang, D\. Almeida, C\. Wainwright, P\. Mishkin, C\. Zhang, S\. Agarwal, K\. Slama, A\. Ray,et al\.\(2022\)Training language models to follow instructions with human feedback\.Advances in neural information processing systems35,pp\. 27730–27744\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1)\.
- J\. Park, J\. Lee, D\. Yoo, and I\. S\. Kweon \(2018\)Distort\-and\-recover: color enhancement using deep reinforcement learning\.InProceedings of the IEEE conference on computer vision and pattern recognition,pp\. 5928–5936\.Cited by:[§D\.1\.2](https://arxiv.org/html/2605.29267#A4.SS1.SSS2.p2.1)\.
- J\. Perdomo, T\. Zrnic, C\. Mendler\-Dünner, and M\. Hardt \(2020\)Performative prediction\.InInternational Conference on Machine Learning,pp\. 7599–7609\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p1.1),[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p2.1),[§3](https://arxiv.org/html/2605.29267#S3.p3.1)\.
- G\. Piliouras and F\. Yu \(2023\)Multi\-agent performative prediction: from global stability and optimality to chaos\.InProceedings of the 24th ACM Conference on Economics and Computation,pp\. 1047–1074\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1)\.
- J\. D\. Pressman, K\. Crowson, and S\. C\. Contributors \(2022\)Simulacra aesthetic captions\.Technical reportTechnical ReportVersion 1\.0,Stability AI\.External Links:[Link](https://github.com/JD-P/simulacra-aesthetic-captions)Cited by:[§1](https://arxiv.org/html/2605.29267#S1.p3.1)\.
- R\. Rafailov, A\. Sharma, E\. Mitchell, C\. D\. Manning, S\. Ermon, and C\. Finn \(2023\)Direct preference optimization: your language model is secretly a reward model\.Advances in neural information processing systems36,pp\. 53728–53741\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1),[item 3](https://arxiv.org/html/2605.29267#S2.I1.i3.p1.10)\.
- V\. Raheja, D\. Kumar, R\. Koo, and D\. Kang \(2023\)CoEdIT: text editing by task\-specific instruction tuning\.InFindings of the Association for Computational Linguistics: EMNLP 2023,Singapore,pp\. 5274–5291\.External Links:[Link](https://aclanthology.org/2023.findings-emnlp.350/),[Document](https://dx.doi.org/10.18653/v1/2023.findings-emnlp.350)Cited by:[§D\.2](https://arxiv.org/html/2605.29267#A4.SS2.p6.11)\.
- Reuters \(2024\)Reuters and AI\.Note:[https://www\.reuters\.com/info\-pages/reuters\-and\-ai](https://www.reuters.com/info-pages/reuters-and-ai)October 30, 2024\. Accessed: 2026\-05\-16Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p8.1)\.
- O\. Ronneberger, P\. Fischer, and T\. Brox \(2015\)U\-net: convolutional networks for biomedical image segmentation\.InInternational Conference on Medical image computing and computer\-assisted intervention,pp\. 234–241\.Cited by:[§D\.1\.1](https://arxiv.org/html/2605.29267#A4.SS1.SSS1.p1.11)\.
- C\. Schuhmann, R\. Beaumont, R\. Vencu, C\. Gordon, R\. Wightman, M\. Cherti, T\. Coombes, A\. Katta, C\. Mullis, M\. Wortsman,et al\.\(2022\)Laion\-5b: an open large\-scale dataset for training next generation image\-text models\.Advances in neural information processing systems35,pp\. 25278–25294\.Cited by:[§1](https://arxiv.org/html/2605.29267#S1.p3.1)\.
- M\. E\. A\. Seddik, S\. Chen, S\. Hayou, P\. Youssef, and M\. A\. DEBBAH \(2024\)How bad is training on synthetic data? a statistical analysis of language model collapse\.InFirst Conference on Language Modeling,External Links:[Link](https://openreview.net/forum?id=t3z6UlV09o)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- I\. Shumailov, Z\. Shumaylov, Y\. Zhao, N\. Papernot, R\. Anderson, and Y\. Gal \(2024\)AI models collapse when trained on recursively generated data\.Nature631\(8022\),pp\. 755–759\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p2.1)\.
- J\. Song, C\. Meng, and S\. Ermon \(2021\)Denoising diffusion implicit models\.InInternational Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=St1giarCHLP)Cited by:[§D\.1\.1](https://arxiv.org/html/2605.29267#A4.SS1.SSS1.p1.11)\.
- K\. Sun, J\. Pan, Y\. Ge, H\. Li, H\. Duan, X\. Wu, R\. Zhang, A\. Zhou, Z\. Qin, Y\. Wang,et al\.\(2023\)Journeydb: a benchmark for generative image understanding\.Advances in neural information processing systems36,pp\. 49659–49678\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1),[Appendix B](https://arxiv.org/html/2605.29267#A2.p7.1)\.
- X\. Tan, Z\. Hu, and X\. Zhang \(2026\)Benchmarking bias mitigation toward fairness without harm from vision to LVLMs\.InThe Fourteenth International Conference on Learning Representations,External Links:[Link](https://openreview.net/forum?id=GLPmZhhCAE)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- R\. Taori, I\. Gulrajani, T\. Zhang, Y\. Dubois, X\. Li, C\. Guestrin, P\. Liang, and T\. B\. Hashimoto \(2023\)Stanford alpaca: an instruction\-following llama model\.GitHub\.Note:[https://github\.com/tatsu\-lab/stanford\_alpaca](https://github.com/tatsu-lab/stanford_alpaca)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[Appendix B](https://arxiv.org/html/2605.29267#A2.p7.1),[§1](https://arxiv.org/html/2605.29267#S1.p1.1)\.
- R\. Taori and T\. Hashimoto \(2023\)Data feedback loops: model\-driven amplification of dataset biases\.InInternational Conference on Machine Learning,pp\. 33883–33920\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1)\.
- Q\. Team \(2024\)Qwen2\.5: a party of foundation models\.External Links:[Link](https://qwenlm.github.io/blog/qwen2.5/)Cited by:[§D\.2](https://arxiv.org/html/2605.29267#A4.SS2.p5.3),[§5](https://arxiv.org/html/2605.29267#S5.p4.8)\.
- The Associated Press \(2024\)Updates to Generative AI Standards\.Note:[https://www\.ap\.org/the\-definitive\-source/behind\-the\-news/updates\-to\-generative\-ai\-standards](https://www.ap.org/the-definitive-source/behind-the-news/updates-to-generative-ai-standards)May 8, 2024\. Accessed: 2026\-05\-16Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p8.1)\.
- P\. von Platen, S\. Patil, A\. Lozhkov, P\. Cuenca, N\. Lambert, K\. Rasul, M\. Davaadorj, D\. Nair, S\. Paul, W\. Berman, Y\. Xu, S\. Liu, and T\. Wolf \(2022\)Diffusers: state\-of\-the\-art diffusion models\.GitHub\.Note:[https://github\.com/huggingface/diffusers](https://github.com/huggingface/diffusers)Cited by:[§D\.1\.1](https://arxiv.org/html/2605.29267#A4.SS1.SSS1.p1.11),[§5](https://arxiv.org/html/2605.29267#S5.p3.8)\.
- Y\. Wang, Z\. Cai, Y\. Bao, X\. Zhang, and Y\. Liu \(2026\)Observations and remedies for large language model bias in self\-consuming performative loop\.arXiv preprint arXiv:2601\.05184\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p2.1)\.
- Y\. Wang, Y\. Kordi, S\. Mishra, A\. Liu, N\. A\. Smith, D\. Khashabi, and H\. Hajishirzi \(2023\)Self\-instruct: aligning language models with self\-generated instructions\.InProceedings of the 61st annual meeting of the association for computational linguistics \(volume 1: long papers\),pp\. 13484–13508\.Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p7.1)\.
- Z\. Wang, Z\. Wu, Y\. Zhang, X\. Guan, N\. Jain, Q\. Lu, S\. Gupta, and A\. Koshiyama \(2025\)Bias amplification: large language models as increasingly biased media\.InProceedings of the 14th International Joint Conference on Natural Language Processing and the 4th Conference of the Asia\-Pacific Chapter of the Association for Computational Linguistics,pp\. 115–132\.Cited by:[Appendix B](https://arxiv.org/html/2605.29267#A2.p8.1)\.
- X\. Wei and X\. Zhang \(2025\)Self\-consuming generative models with adversarially curated data\.InForty\-second International Conference on Machine Learning,External Links:[Link](https://openreview.net/forum?id=UWWNxyIT1h)Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p2.1),[§1](https://arxiv.org/html/2605.29267#S1.p2.1),[§2](https://arxiv.org/html/2605.29267#S2.p4.1)\.
- S\. Wyllie, I\. Shumailov, and N\. Papernot \(2024\)Fairness feedback loops: training on synthetic data amplifies bias\.InProceedings of the 2024 ACM Conference on Fairness, Accountability, and Transparency,pp\. 2113–2147\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[§1](https://arxiv.org/html/2605.29267#S1.p2.1)\.
- T\. Xie and X\. Zhang \(2024\)Automating data annotation under strategic human agents: risks and potential solutions\.Advances in Neural Information Processing Systems37,pp\. 127436–127482\.Cited by:[§1](https://arxiv.org/html/2605.29267#S1.p2.1)\.
- T\. Xie, D\. Zhu, J\. Liu, M\. Khalili, and X\. Zhang \(2025\)SPRINT: stochastic performative prediction with variance reduction\.arXiv preprint arXiv:2509\.17304\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p3.1)\.
- Q\. Yu, Q\. Sun, X\. Zhang, Y\. Cui, F\. Zhang, Y\. Cao, X\. Wang, and J\. Liu \(2024\)Capsfusion: rethinking image\-text data at scale\.InProceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition,pp\. 14022–14032\.Cited by:[Appendix A](https://arxiv.org/html/2605.29267#A1.p1.1),[Appendix B](https://arxiv.org/html/2605.29267#A2.p7.1),[§1](https://arxiv.org/html/2605.29267#S1.p1.1)\.
- R\. Zhang, L\. Guo, S\. Huang, and B\. Wen \(2021\)Rellie: deep reinforcement learning for customized low\-light image enhancement\.InProceedings of the 29th ACM international conference on multimedia,pp\. 2429–2437\.Cited by:[§D\.1\.2](https://arxiv.org/html/2605.29267#A4.SS1.SSS2.p2.1)\.
- H\. Zhao, J\. Fu, and T\. Pham \(2025\)Convergence and stability analysis of self\-consuming generative models with heterogeneous human curation\.arXiv preprint arXiv:2511\.09002\.Cited by:[§1](https://arxiv.org/html/2605.29267#S1.p2.1),[§2](https://arxiv.org/html/2605.29267#S2.p4.1)\.
- Y\. Zhao \(2022\)Optimizing the performative risk under weak convexity assumptions\.InOPT 2022: Optimization for Machine Learning \(NeurIPS 2022 Workshop\),External Links:[Link](https://openreview.net/forum?id=Ut_vApkulkk)Cited by:[§B\.1](https://arxiv.org/html/2605.29267#A2.SS1.p2.1)\.

## Appendix ARelated work

Self\-consuming Training on Synthetic Data\.A rapid growing body of work studies iterative retraining on model\-generated data from theoretical or empirical perspectives\. The self\-consuming training loops can lead to several degradation phenomena\.Shumailovet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib11)\)show that indiscriminate recursive training can cause irreversible defects and disappearance of distributional tails\.Alemohammadet al\.\([2023](https://arxiv.org/html/2605.29267#bib.bib12)\)analyze “autophagous” loops and show systematic quality\-diversity deterioration without sufficient fresh real data, formalized as Model Autophagy Disorder \(MAD\)\. At the same time, “collapse” is not universal:Bertrandet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib9)\)provide conditions under which iterative retraining on mixtures of real and synthetic data can be stable, including empirical validation on diffusion models\.Gerstgrasseret al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib37)\); Seddiket al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib47)\)both argue that accumulating synthetic data alongside real data can mitigate collapse\. Theoretical refinements examine regimes of strong collapse and how model capacity and mixture proportions affect the bias\-variance decomposition of errors\(Dohmatobet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib38)\)\.Fuet al\.\([2025](https://arxiv.org/html/2605.29267#bib.bib14)\)theoretically study how both model architecture and the proportion between real and synthetic data influence recursive training loops\. Meanwhile, self\-consuming training induces bias amplification\. Model\-induced distribution shifts can amplify unfairness, andWyllieet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib20)\)analyze fairness feedback loops over generations and propose “algorithmic reparation” as a corrective mechanism\. In language,Wanget al\.\([2026](https://arxiv.org/html/2605.29267#bib.bib18)\)show that iterative retraining on synthetic data coupled with deployment\-driven feedback can systematically amplify preference bias\. In vision,Chenet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib41)\)evaluate the amplifying effect of self\-consuming training on synthetic data on social biases\. Complementary to studies of bias amplification under recursive training, recent benchmark work\(Tanet al\.,[2026](https://arxiv.org/html/2605.29267#bib.bib42)\)emphases the need for harm\-aware fairness evaluation in vision and multimodal models\. Moreover, when systems amplify bias,Taori and Hashimoto \([2023](https://arxiv.org/html/2605.29267#bib.bib40)\)formalize data feedback loops in conditional prediction and connect stability to calibration\-like properties\. Furthermore, extending to multi\-model self\-consuming systems,Gao and Li \([2025](https://arxiv.org/html/2605.29267#bib.bib8)\)analyze the co\-evolving dynamic while modeling the system with simplified Gaussian distribution\.Huet al\.\([2025](https://arxiv.org/html/2605.29267#bib.bib7)\)empirically explore the multi\-model collapse pattern for recursive generate\-train loops\. However, while self\-consuming training in multi\-model systems is quite common\(Taoriet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib27); Yuet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib28); Shumailovet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib11)\), its general evolution stability remains unclear, which is one of the objectives of our explorations\.

Human Curation and Model Alignment\.Real\-world synthetic data pipelines typically include user curation\. For example, the JourneyDB dataset contains images generated and human\-curated originating from the Midjourney Discord\(Sunet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib43)\)\.Fenget al\.\([2025](https://arxiv.org/html/2605.29267#bib.bib44)\)study verifier\-based selection on synthetic data can prevent collapse even when generation is imperfect\.Ferbachet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib6)\)show that human curation can be regarded as an implicit preference optimization mechanism, and iterative retraining can improve expected rewards under suitable curation assumptions\. In contrast, if the curation is adversarial, model alignment will be disrupted\(Wei and Zhang,[2025](https://arxiv.org/html/2605.29267#bib.bib10)\)\. For preference optimization and alignment, RLHF is a widely used technique to align models to human preference\(Christianoet al\.,[2017](https://arxiv.org/html/2605.29267#bib.bib45); Ouyanget al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib46)\)\. Direct Preference Optimization\(Rafailovet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib33)\), as its variant, shows that preference alignment can be achieved via a simple classification\-style objective without explicit modeling\. The lack of analysis on the impact of curation on preference alignment in multi\-model self\-consuming systems forms the basis of our work\.

Performative Prediction\.Similar to the stability results fromBertrandet al\.\([2024](https://arxiv.org/html/2605.29267#bib.bib9)\), our stability analysis in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6)is related to the literature on “performative prediction”\(Perdomoet al\.,[2020](https://arxiv.org/html/2605.29267#bib.bib29)\)\. Performative prediction formalizes settings in which the model outputs affect the data\-generating process, yielding a retraining dynamic whose fixed points correspond to performative stable solutions\(Xieet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib3); Jinet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib5),[2026](https://arxiv.org/html/2605.29267#bib.bib4)\)\. Self\-consuming and model interactions are also included in this framework\.Perdomoet al\.\([2020](https://arxiv.org/html/2605.29267#bib.bib29)\)provide a foundation for treating iterative training as interacting with a decision\-dependent distribution map–precisely the lens adopted in our multi\-model evolution setting\. Further work\(Brownet al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib30)\)generalizes the framework by allowing the induced distribution to depend on an evolving system state, thereby capturing path dependence and multi\-stability\. Moving beyond stylized convex objective,Mofakhamiet al\.\([2023](https://arxiv.org/html/2605.29267#bib.bib31)\)extend stability and convergence analyses to nonconvex regimes and highlights that stability is governed by the sensitivity or predictions to deployment\. Meanwhile, multi\-agent performative prediction captures coupled learning dynamics where multiple decision makers jointly influence and are influenced by the evolving data distribution\.Piliouras and Yu \([2023](https://arxiv.org/html/2605.29267#bib.bib36)\)characterize regimes ranging from global stability to instability and even chaos for the multi\-agent system\.Liet al\.\([2022](https://arxiv.org/html/2605.29267#bib.bib35)\)develop a decentralized, coupled viewpoint in which multiple agents jointly influence data distribution and study convergence under greedy deployment and consensus mechanisms\.

## Appendix BDiscussion

Our framework provides a mechanistic and analyzable understanding of multi\-model self\-consuming ecosystem’s long\-term evolution, stability and its preference alignment\. Beyond the main theoretical and experiment results, we highlight several extensions and implications, and then discuss the limitations of the present analysis\.

Preference alignment along the training trajectory\.Our curation\-effect analysis focuses on how alignment metrics change around the stable point after convergence, since analysis at equilibrium isolates the system\-level externalities that persist after convergence caused by model interaction\. The same framework can be extended to analyze alignment metrics during the iterative process by replacing the fixed evaluation marginal distribution with the iteration\-dependent induced marginal distribution\. In this case,Jp​\(θt\)=𝔼y∼ℰθ,x∼pθt​\(x\|y\)​rθ​\(x,y\)→Jp​\(θt,ϕt\)=𝔼y∼Py​\(θt,ϕt\),x∼pθt​\(x\|y\)​rθ​\(x,y\)J\_\{p\}\(\\theta\_\{t\}\)=\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta\_\{t\}\}\(x\|y\)\}r\_\{\\theta\}\(x,y\)\\rightarrow J\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)=\\mathbb\{E\}\_\{y\\sim P^\{y\}\(\\theta\_\{t\},\\phi\_\{t\}\),x\\sim p\_\{\\theta\_\{t\}\}\(x\|y\)\}r\_\{\\theta\}\(x,y\)\. Therefore,JpJ\_\{p\}depends not only on model parameterθt\\theta\_\{t\}but also on parameters of models that interact with modelθ\\theta\. The remaining derivations, such as∂Jp​\(θt,ϕt\)∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and decomposition of self\-influence and cross\-influence, can be completed following the current framework\.

NN\-model framework\.We focus on the two\-model system as the minimal unit that already reveals the core mechanism of system stability and how human curation affect the preference alignments\. Human curation is not a per\-model improvement in isolation, but an intervention on a coupled dynamic system where cross\-model influences can dominate\. Extending the framework toNNmodels \(N\>2N\>2\) is conceptually straightforward in terms of block\-structured sensitivities since model interaction is achieved by linearly mixing the data generated by different models in the training dataset, but introduces additional technical challenges, such as coupling\-graph topology and more complex equilibrium structure\. We do not attempt to resolve here, and we expect theory\-related phenomenons to remain relevant in larger ecosystems, and potentially more pronounced as indirect influence paths proliferate\.

Preference domain mismatch \(PDM\)\.PDM highlights that coupling\-induced preference conflicts can be weakly reflected, or even appear absent under a single alignment evaluation metric when the induced training distribution shifts into regions not well covered by the evaluation domain\. We expect such mismatch to be common in practice, particularly when the training data is acquired or filtered from heterogeneous internet corpora, where selection pipelines may fail to capture the relevant preference data or may emphasize preferences that are effectively orthogonal to the target objective\. Importantly, PDM should not be regarded as ”guaranteeing safety” of internet\-sourced synthetic data, but it explains how preference conflicts and distributional shifts may remain invisible to a narrow alignment evaluation metric\. The internet\-sourced synthetic data can appear ”safe” under that metric while still inducing distributional shifts\.

Practical implications: local diagnostics and interventions\.Our theory is not primarily targeted to precisely predict end\-to\-end training outcomes, but to provide local diagnostics and intervention rules for coupled self\-consuming ecosystems\. By estimating the local and cross\-model responses \(Sp,Sq,Cp,CqS\_\{p\},S\_\{q\},C\_\{p\},C\_\{q\}\) together with the sign and decomposition of the curation derivatives \(∂Jp​\(θ\)∂λℋθ,∂Jq​\(ϕ\)∂λℋϕ\\frac\{\\partial J\_\{p\}\(\\theta\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\},\\frac\{\\partial J\_\{q\}\(\\phi\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\}\), self\-influence can be separated from cross\-model propagation, and whether an increase in curation is likely to be neutralized or even reversed by ecosystem externalities and model interactions can be assessed without long\-time simulation\. In practice, these quantities can be approximated via local perturbation experiments, and serve as local indicators of stability and preference transmission\. When cross\-influence terms dominate or the predicted derivative sign contradicts the single\-model intuition, they provide an early warning that preference conflicts are being amplified, motivating targeted mitigations such as improving the real data ratios, reducing cross\-sourced curation, or isolating coupling pathways\. However, as discussed in Section[D\.3](https://arxiv.org/html/2605.29267#A4.SS3), precise estimation of these matrices is difficult for large models\. Both the assumption\-level quantities and the Section[4](https://arxiv.org/html/2605.29267#S4)matrices are affected by sampling noisse, finite\-sample statistical error, and approximation error from large\-scale high\-order matrix estimation\. The development of accurate and scalable estimators for large models is also an important future research direction\(Basuet al\.,[2021](https://arxiv.org/html/2605.29267#bib.bib17)\)\.

Real world examples\.We provide several examples of where the problem discussed in this paper arises in practice, some of which further elaborate the examples in Section[1](https://arxiv.org/html/2605.29267#S1)\. A first example is modern alignment and safety pipelines, where preference signals are explicitly heterogeneous\. Safe RLHF\(Daiet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib24)\)formulates a tension between helpfulness and harmlessness, while Safety\-Tuned LLaMAs\(Bianchiet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib25)\)shows that emphasizing helpfulness alone can produce unsafe behavior, whereas stronger safety tuning can induce exaggerated refusals on benign inputs\. This is precisely the kind of naturally occurring preference conflict that motivates our multi\-model view: different models in the pipeline have different preferences \(helpfulness/harmlessness\), and their interaction can shape the final model behavior in nontrivial ways\.

A second example is instruction\-tuning/model\-distillation pipelines such as Self\-Instruct\(Wanget al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib26)\)and Alpaca\(Taoriet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib27)\), where one language model generates instruction\-response data, the outputs are filtered/curated, and another model is then fine\-tuned on the resulting synthetic data\. In such settings, the relevant preference differences are not artificial: they can correspond to natural trade\-offs such as concise vs\. detailed responses, style/formality preferences, or general helpfulness vs\. safety\-oriented behavior\. A third example is web\-scale multimodal data construction\. CapsFusion\(Yuet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib28)\)uses LLMs to consolidate and refine web image\-text pairs and image\-only data into synthetic captions for future multimodal model training, while datasets such as JourneyDB\(Sunet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib43)\)further illustrate that large\-scale generated image–text ecosystems already involve content\- and style\-sensitive curation\. In such settings, natural preference differences can arise between literal captioning vs\. richer descriptive captioning, style fidelity vs\. semantic coverage, or aesthetic preference vs\. downstream task utility\.

Another realistic case is AI assisted news production in a politically polarized media ecosystem\. Different outlets serve audiences with systematically different political leanings \(right/liberal\), while major news organizations such as AP and Reuters publicly state that generative AI is already used in news workflows, including summaries, headlines, writing, editing, and publishing\(The Associated Press,[2024](https://arxiv.org/html/2605.29267#bib.bib21); Reuters,[2024](https://arxiv.org/html/2605.29267#bib.bib22)\)\. In such an environment, different outlets may naturally prefer different political framings of the same event; for example, emphasizing border security and law\-and\-order versus civil rights and inclusion\. If AI tools are used to draft, summarize, rewrite, or prioritize content under these different editorial preferences, then even mild framing differences can affect which narrative preferences are preserved, amplified, and reused downstream\. This concern is also consistent with prior work\(Wanget al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib39)\), which shows that recursive synthetic training can amplify political bias across generations\.

### B\.1Limitations of our framework

Finite samples’ statistical error\.Our theoretical analysis does not explicitly quantify finite\-sample effects, i\.e\., the discrepancy between the empirical distribution induced by a finite training set and the population \(or iteration\-dependent induced\) distribution\. Such discrepancies are unavoidable in both our experiments and real\-world deployments\. Prior work suggests that their impact can be mitigated by increasing the amount of training data \(thereby reducing estimation error\) or by improving algorithmic stability through conservative step\-size choices and learning\-rate schedules, among other standard practices\(Perdomoet al\.,[2020](https://arxiv.org/html/2605.29267#bib.bib29); Hardtet al\.,[2016](https://arxiv.org/html/2605.29267#bib.bib51); Cutleret al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib52)\)\. A more complete treatment should incorporate empirical estimation error bounds and stability analysis considering the stochastic optimization noise and training data distribution shifts due to self\-consuming\. We view this as complementary to our theory results, and it would quantify when stability conclusions are robust under sampling noises\.

Theory assumptions\.Following prior work on iterative self\-consuming retraining and performative prediction\(Perdomoet al\.,[2020](https://arxiv.org/html/2605.29267#bib.bib29); Bertrandet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib9); Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\), we adopt standard regularity assumptions \(Assumption[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)–[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)\) that ensure \(i\) a stable learning update \(e\.g\., smoothness and a well\-behaved optimization landscape\) and \(ii\) controlled sensitivity of the induced training distribution to model changes\. These assumptions enable characterizations of stability and local response in our coupled retraining dynamics\. While these assumptions facilitate a clean first\-step analysis of multi\-model self\-consuming loops in our paper, similar to prior work’s limitations, they can be difficult to verify for modern over\-parameterized neural networks and may not fully reflect the nonconvex optimization landscape encountered in practice\. Encouragingly, recent work has begun relaxing these requirements, e\.g\., by establishing guarantees under smooth nonconvex objectives or by replacing parameter\-space regularity with prediction\-level stability conditions that better align with neural network models\(Mofakhamiet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib31); Li and Wai,[2024](https://arxiv.org/html/2605.29267#bib.bib54); Zhao,[2022](https://arxiv.org/html/2605.29267#bib.bib53)\)\. Extending these more general analyses to strongly coupled multi\-model self\-consuming systems remains challenging, since model interaction introduces coupled feedback pathways and more complex equilibrium structures that are absent in single\-model settings\. In particular, coupling can also propagate local estimation errors across models and iterations\.

Experiments\.To cleanly expose coupling\-induced preference conflicts, since our theory mainly depends on self and cross influences and preference\-direction alignment, not on the particular rewards, we adopt controlled reward constructions that may produce “failure modes” which are consistent with previous conclusion\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\)\. For example, model outputs visually degenerate images under extreme preferences, such as the almost monochromatic images shown in[Figure8](https://arxiv.org/html/2605.29267#A4.F8)\. We treat our experiment setups as stress tests and existence proofs\. Observing non\-monotonicity that enhancing curation strength does not improve preference alignment in a simplified environment with explicit preferences and strong coupling is sufficient to prove that monotonic improvement from increased curation cannot be assumed as a general principle\. Moreover, current experiment results also raise a theoretical question:In a self\-consuming ecosystem with multi\-model interactions, how can we characterize the distance between the stable point of such system after convergence with human curation, and the optimal point training with only real data?This problem has been solved in the single\-model scenario\(Bertrandet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib9); Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\), but it remains unknown for multi\-model systems\.

We view exploring the extension of our framework and addressing these limitations as promising directions for future work\.

## Appendix CConclusion

This paper studies the long\-term evolution and preference alignment of the multi\-model ecosystem in self\-consuming training loops\. We theoretically analyze the stability conditions in such system under multi\-model interactions\. Building on this foundation, we examine how human curation affects preference alignment when multiple, potentially heterogeneous preferences coexist in the multi\-model system, disentangling both self\-influence and cross\-influence transmitted to other models through model interactions\. Our findings indicate that the preference alignment of one single model is jointly shaped by self\-consuming loops and model interactions, and highlight that increasing the curation strength does not necessarily improve preference alignment\.

## Appendix DExperiment Discussion And Additional Experiments

### D\.1Experiments on CIFAR\-10 datasets

In this section, we present the details of the experiments in the main paper\. We first detail the model architecture and training settings in[SectionD\.1\.1](https://arxiv.org/html/2605.29267#A4.SS1.SSS1)\. Next, the formal definition of rewardsrθ,rϕr\_\{\\theta\},r\_\{\\phi\}are shown in[SectionD\.1\.2](https://arxiv.org/html/2605.29267#A4.SS1.SSS2)\. Finally, we show more detailed experiments in[SectionD\.1\.3](https://arxiv.org/html/2605.29267#A4.SS1.SSS3)\.

#### D\.1\.1Settings

Model architecture\.We implement the two class\-conditional diffusion modelsθ\\theta,ϕ\\phiusing theUNet2DModelarchitectures\(Ronnebergeret al\.,[2015](https://arxiv.org/html/2605.29267#bib.bib61)\)from the Hugging Face Diffusers library\(von Platenet al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib60)\)\. The networks are both 4\-level 2D UNet that maps a noisy RGBx∈ℝ3×32×32x\\in\\mathbb\{R\}^\{3\\times 32\\times 32\}and timestepttto an output of the same shape\. We setb​l​o​c​k​\_​o​u​t​\_​c​h​a​n​e​e​l​s=\(92,192,192,384\)block\\\_out\\\_chaneels=\(92,192,192,384\)with one ResNet layer per block \(l​a​y​e​r​s​\_​p​e​r​\_​b​l​o​c​k=1layers\\\_per\\\_block=1\)\. Attention blocks are used at the intermediate spatial resolutions \(16×1616\\times 16and8×88\\times 8for32×3232\\times 32inputs\), while the highest and loweset resolutions are standard ResNet blocks\. We train them with a DDPM noise scheduler and for sampling, we use DDIM instantiated from DDPM configuration\(Hoet al\.,[2020](https://arxiv.org/html/2605.29267#bib.bib59); Songet al\.,[2021](https://arxiv.org/html/2605.29267#bib.bib58)\)\. Class conditioning is provided via integer labels0∼90\\sim 9, whose embeddings are added to the timestep embeddings in the UNet forward pass\. All unspecified UNet hyperparameters follow the Diffusers defaults\.

Iterative retraining process\.[Algorithm1](https://arxiv.org/html/2605.29267#alg1)describes the process of iterative retraining on mixture datasets for the two\-model interacting system, and modelθ\\thetaandϕ\\phiare updated synchronously in our CIFAR\-10 experiments\. If models are updated asynchronously, the algorithm for asynchronous updates can be obtained by replacing the simultaneous updates of the two models in[Algorithm1](https://arxiv.org/html/2605.29267#alg1)with sequential updates, and ensuring that the model parameters for cross\-model samples are adjusted accordingly\.

Evaluation setting\.For modelθ\\thetaandϕ\\phi, which share the same model architecture in our experiment, we use a fixed set of 5k CIFAR\-10 data samples as the common evaluation dataset to compute empirical estimates ofJp​\(θt\)J\_\{p\}\(\\theta\_\{t\}\)andJq​\(ϕt\)J\_\{q\}\(\\phi\_\{t\}\)\.

Algorithm 1Iterative retraining of two\-model ecosystem under synchronous updates with model interactionsInput:Real data

ℛθ=\{\(x,y\)\}\\mathcal\{R\}^\{\\theta\}=\\\{\(x,y\)\\\}where

xxis the image and

yyis the label, and

ℛϕ\\mathcal\{R\}^\{\\phi\}, reward functions

rθr\_\{\\theta\}and

rϕr\_\{\\phi\}, learning procedures

𝒜θ\\mathcal\{A\}\_\{\\theta\}and

𝒜ϕ\\mathcal\{A\}\_\{\\phi\}
Param:For model

θ\\theta: training data size

NθN\_\{\\theta\}, real data ratio

λℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}, synthetic data ratio

λ𝒮θ\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}, synthetic curated data ratio

λℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}, cross\-model data ratio

λθϕ\\lambda\_\{\\theta\}^\{\\phi\}\. Symmetrically, for model

ϕ\\phi: training data size

NϕN\_\{\\phi\}, real data ratio

λℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}, synthetic data ratio

λ𝒮ϕ\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\}, synthetic curated data ratio

λℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}, cross\-model data ratio

λϕθ\\lambda\_\{\\phi\}^\{\\theta\}\. Iteration number

TT\.

θ0=𝒜θ​\(ℛθ\)\\theta\_\{0\}=\\mathcal\{A\}\_\{\\theta\}\(\\mathcal\{R\}^\{\\theta\}\),

ϕ0=𝒜ϕ​\(ℛϕ\)\\phi\_\{0\}=\\mathcal\{A\}\_\{\\phi\}\(\\mathcal\{R\}^\{\\phi\}\),

𝒟0θ=ℛθ\\mathcal\{D\}\_\{0\}^\{\\theta\}=\\mathcal\{R\}^\{\\theta\},

𝒟0ϕ=ℛϕ\\mathcal\{D\}\_\{0\}^\{\\phi\}=\\mathcal\{R\}^\{\\phi\}
for

t=1t=1to

TTdo

𝒮tθ=\{\(xi,yi\)\}i=1Nθ​λ𝒮θ​\(1−λθϕ\)∪\{\(x^i,y^i\)\}i=1Nθ​λ𝒮θ​λθϕ\\mathcal\{S\}\_\{t\}^\{\\theta\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\_\{\\theta\}\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\}\\cup\\\{\(\\widehat\{x\}\_\{i\},\\widehat\{y\}\_\{i\}\)\\\}\_\{i=1\}^\{N\_\{\\theta\}\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\\lambda\_\{\\theta\}^\{\\phi\}\}where

yi∼𝒟t−1θ,y,xi∼pθt−1​\(x\|yi\)y\_\{i\}\\sim\\mathcal\{D\}\_\{t\-1\}^\{\\theta,y\},x\_\{i\}\\sim p\_\{\\theta\_\{t\-1\}\}\(x\|y\_\{i\}\)and

y^i∼𝒟t−1ϕ,y,x^i∼qϕt−1​\(x\|y^i\)\\widehat\{y\}\_\{i\}\\sim\\mathcal\{D\}\_\{t\-1\}^\{\\phi,y\},\\widehat\{x\}\_\{i\}\\sim q\_\{\\phi\_\{t\-1\}\}\(x\|\\widehat\{y\}\_\{i\}\), and

𝒟t−1θ,y,𝒟t−1ϕ,y\\mathcal\{D\}\_\{t\-1\}^\{\\theta,y\},\\mathcal\{D\}\_\{t\-1\}^\{\\phi,y\}are label

yy’s empirical marginal distribution of

𝒟t−1θ,𝒟t−1ϕ\\mathcal\{D\}\_\{t\-1\}^\{\\theta\},\\mathcal\{D\}\_\{t\-1\}^\{\\phi\}, respectively\.

𝒮tϕ=\{\(xi,yi\)\}i=1Nϕ​λ𝒮ϕ​\(1−λϕθ\)∪\{\(x^i,y^i\)\}i=1Nϕ​λ𝒮ϕ​λϕθ\\mathcal\{S\}\_\{t\}^\{\\phi\}=\\\{\(x\_\{i\},y\_\{i\}\)\\\}\_\{i=1\}^\{N\_\{\\phi\}\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\}\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\}\\cup\\\{\(\\widehat\{x\}\_\{i\},\\widehat\{y\}\_\{i\}\)\\\}\_\{i=1\}^\{N\_\{\\phi\}\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\}\\lambda\_\{\\phi\}^\{\\theta\}\}where

y^i∼𝒟t−1θ,y,x^i∼pθt−1​\(x\|y^i\)\\widehat\{y\}\_\{i\}\\sim\\mathcal\{D\}\_\{t\-1\}^\{\\theta,y\},\\widehat\{x\}\_\{i\}\\sim p\_\{\\theta\_\{t\-1\}\}\(x\|\\widehat\{y\}\_\{i\}\)and

yi∼𝒟t−1ϕ,y,xi∼qϕt−1​\(x\|yi\)y\_\{i\}\\sim\\mathcal\{D\}\_\{t\-1\}^\{\\phi,y\},x\_\{i\}\\sim q\_\{\\phi\_\{t\-1\}\}\(x\|y\_\{i\}\)
for

j=1j=1to

Nθ​λℋθN\_\{\\theta\}\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}do

if

j<Nθ​λℋθ​\(1−λθϕ\)j<N\_\{\\theta\}\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)then

y^j∼𝒟t−1θ,y\\widehat\{y\}\_\{j\}\\sim\\mathcal\{D\}\_\{t\-1\}^\{\\theta,y\},

x1,…,xK∼pθt−1​\(x\|y^j\)x\_\{1\},\.\.\.,x\_\{K\}\\sim p\_\{\\theta\_\{t\-1\}\}\(x\|\\widehat\{y\}\_\{j\}\)
xk,1≤k≤Kx\_\{k\},1\\leq k\\leq Kis selected based on Eq\. \([3](https://arxiv.org/html/2605.29267#S2.E3)\) using reward

rθr\_\{\\theta\}and

x^j←xk\\widehat\{x\}\_\{j\}\\leftarrow x\_\{k\}\{curated synthetic data generated from model

θ\\theta\}

else

y^j∼𝒟t−1ϕ,y\\widehat\{y\}\_\{j\}\\sim\\mathcal\{D\}\_\{t\-1\}^\{\\phi,y\},

x1,…,xK∼qϕt−1​\(x\|y^j\)x\_\{1\},\.\.\.,x\_\{K\}\\sim q\_\{\\phi\_\{t\-1\}\}\(x\|\\widehat\{y\}\_\{j\}\)
xk,1≤k≤Kx\_\{k\},1\\leq k\\leq Kis selected based on Eq\. \([3](https://arxiv.org/html/2605.29267#S2.E3)\) using reward

rϕr\_\{\\phi\}and

x^j←xk\\widehat\{x\}\_\{j\}\\leftarrow x\_\{k\}\{cross\-model curated synthetic data generated from model

ϕ\\phi\}

endif

endfor

ℋtθ=\{\(x^j,y^j\)\}j=1Nθ​λℋθ\\mathcal\{H\}\_\{t\}^\{\\theta\}=\\\{\(\\widehat\{x\}\_\{j\},\\widehat\{y\}\_\{j\}\)\\\}\_\{j=1\}^\{N\_\{\\theta\}\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}

Compute

ℋtϕ\\mathcal\{H\}\_\{t\}^\{\\phi\}symmetrically

𝒟tθ=𝒮tθ∪ℋtθ∪\{\(xi,yi\)∈ℛθ\}i=1Nθ​λℛθ\\mathcal\{D\}\_\{t\}^\{\\theta\}=\\mathcal\{S\}\_\{t\}^\{\\theta\}\\cup\\mathcal\{H\}\_\{t\}^\{\\theta\}\\cup\\\{\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{R\}^\{\\theta\}\\\}\_\{i=1\}^\{N\_\{\\theta\}\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\},

𝒟tϕ=𝒮tϕ∪ℋtϕ∪\{\(xi,yi\)∈ℛϕ\}i=1Nϕ​λℛϕ\\mathcal\{D\}\_\{t\}^\{\\phi\}=\\mathcal\{S\}\_\{t\}^\{\\phi\}\\cup\\mathcal\{H\}\_\{t\}^\{\\phi\}\\cup\\\{\(x\_\{i\},y\_\{i\}\)\\in\\mathcal\{R\}^\{\\phi\}\\\}\_\{i=1\}^\{N\_\{\\phi\}\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\}
θt=𝒜θ​\(𝒟tθ\)\\theta\_\{t\}=\\mathcal\{A\}\_\{\\theta\}\(\\mathcal\{D\}\_\{t\}^\{\\theta\}\),

ϕt=𝒜ϕ​\(𝒟tϕ\)\\phi\_\{t\}=\\mathcal\{A\}\_\{\\phi\}\(\\mathcal\{D\}\_\{t\}^\{\\phi\}\)
endfor

#### D\.1\.2Reward design

HSV representation of a pixel\.HSV is a cylindrical reparameterization of RGB intended to align better with intuitive color attributes\. For a pixelu=\(R,G,B\)∈\[0,1\]3u=\(R,G,B\)\\in\[0,1\]^\{3\}of an image, denoteV=max⁡\(R,G,B\)V=\\max\(R,G,B\),m=min⁡\(R,G,B\)m=\\min\(R,G,B\), andΔ=V−m\\Delta=V\-m\. The Value isVV, and the Saturation is

S=\{0,V=0,ΔV\+10−8,V\>0\.S=\\begin\{cases\}0,\\ \\ V=0,\\\\ \\frac\{\\Delta\}\{V\+10^\{\-8\}\},\\ \\ V\>0\.\\end\{cases\}The Hue is

H=\{0,Δ=0,16​\(G−BΔmod6\),V=R,16​\(B−RΔ\+2\),V=G,16​\(R−GΔ\+4\),V=B\.H=\\begin\{cases\}0,\\ \\ \\Delta=0,\\\\ \\frac\{1\}\{6\}\\big\(\\frac\{G\-B\}\{\\Delta\}\\mod 6\\big\),\\ \\ V=R,\\\\ \\frac\{1\}\{6\}\\big\(\\frac\{B\-R\}\{\\Delta\}\+2\\big\),\\ \\ V=G,\\\\ \\frac\{1\}\{6\}\\big\(\\frac\{R\-G\}\{\\Delta\}\+4\\big\),\\ \\ V=B\.\\end\{cases\}
Why we use hue\-derived rewards\.Hue is a standard, explicitly controlled color attribute\. Prior image\-enhancement research explicitly treats hue preservation/correction as a design objective to avoid perceptual hue distortions, including methods that keep hue constant in HSI/HSV\-style processing pipelines and correction schemes designed to be applicable to deep\-learning\-based enhancement\(Kinoshita and Kiya,[2020](https://arxiv.org/html/2605.29267#bib.bib56)\)\. Reinforcement\-learning formulations for image enhancement are well established\(Zhanget al\.,[2021](https://arxiv.org/html/2605.29267#bib.bib55); Parket al\.,[2018](https://arxiv.org/html/2605.29267#bib.bib57)\)and typically define actions as interpretable color/tonal adjustments and optimize policies using hand\-crafted, non\-reference reward signals\. Using a hue\-derived reward is a direct instantiation of this color\-aware reward shaping paradigm\. In our setting, we adopt hue\-based rewards to better illustrate the preference conflicts of different vision models in the experiment\.

Reward Design\.Given a model generated imagex∈\[−1,1\]3×32×32x\\in\[\-1,1\]^\{3\\times 32\\times 32\}, we first map it to\[0,1\]\[0,1\]viax01=c​l​i​p​\(x,−1,1\)\+12x\_\{01\}=\\frac\{clip\(x,\-1,1\)\+1\}\{2\}\. For a pixeluuinx01x\_\{01\}, we compute its HSV representation and obtain its hueH​\(u\)∈\[0,1\)H\(u\)\\in\[0,1\)and saturationS​\(u\)∈\[0,1\]S\(u\)\\in\[0,1\]\. HueHHrepresents a normalized angle on the color wheel \(red–yellow–green–cyan–blue–magenta–red\), while saturationSSmeasures colorfulness\. For near\-gray pixels,S≈0S\\approx 0and hue becomes unstable\. For a given hue intervalℐ⊂\[0,1\)\\mathcal\{I\}\\subset\[0,1\), let

𝟏ℐ​\(t\)=\{1,t∈ℐ,0,otherwise,\\mathbf\{1\}\_\{\\mathcal\{I\}\}\(t\)=\\begin\{cases\}1,\\ \\ t\\in\\mathcal\{I\},\\\\ 0,\\ \\ \\text\{otherwise,\}\\end\{cases\}be the membership indicator\. The hue\-band occupancy score of model\-generated imagexxis:

B​a​n​dℐ​\(x\)=132×32​∑i=132∑j=132𝟏ℐ​\(H​\(x01​\[i,j\]\)\)​S​\(H​\(x01​\[i,j\]\)\)1\.5,Band\_\{\\mathcal\{I\}\}\(x\)=\\frac\{1\}\{32\\times 32\}\\sum\_\{i=1\}^\{32\}\\sum\_\{j=1\}^\{32\}\\mathbf\{1\}\_\{\\mathcal\{I\}\}\\big\(H\(x\_\{01\}\[i,j\]\)\\big\)S\\big\(H\(x\_\{01\}\[i,j\]\)\\big\)^\{1\.5\},wherex01​\[i,j\]x\_\{01\}\[i,j\]is the pixel atiith row,jjth column ofx01x\_\{01\}\. A largerB​a​n​dℐ​\(x\)Band\_\{\\mathcal\{I\}\}\(x\)value indicates that a larger fraction of high\-saturation pixels lies within the target hue band \(ℐ\\mathcal\{I\}\), hence the global appearance is more aligned with the corresponding color tone\.

In our experiment, we map ”warm” and ”cool” tones to two hue intervals on the color wheel\. We implementℐw​a​r​m=\[0\.92,1\)∪\[0,0\.17\]\\mathcal\{I\}\_\{warm\}=\[0\.92,1\)\\cup\[0,0\.17\]covering red\-orange\-yellow, andℐc​o​o​l=\[0\.5,0\.72\]\\mathcal\{I\}\_\{cool\}=\[0\.5,0\.72\]covering cyan\-blue\. Accordingly, we define the warm and cool scores as

W​a​r​m​\(x\)=B​a​n​dℐw​a​r​m​\(x\),C​o​o​l​\(x\)=B​a​n​dℐc​o​o​l​\(x\)\.Warm\(x\)=Band\_\{\\mathcal\{I\}\_\{warm\}\(x\)\},\\ Cool\(x\)=Band\_\{\\mathcal\{I\}\_\{cool\}\(x\)\}\.This choice aligns with common perceptual conventions: red/orange/yellow hues are typically perceived as “warm”, whereas cyan/blue hues are perceived as “cool”\. When a larger portion of high\-saturation pixels falls into the corresponding band, the overall appearance exhibits a stronger warm/cool tone\.

For model generated imagesxx, we define two label\-free rewards consisting of a color preference term \(W​a​r​m​\(x\)Warm\(x\)orC​o​o​l​\(x\)Cool\(x\)\) and a global\-statistics regularizer \(R​\(x\)R\(x\)\) as

rθ​\(x\)=3​W​a​r​m​\(x\)\+0\.3​R​\(x\),rϕ​\(x\)=3​C​o​o​l​\(x\)\+0\.3​R​\(x\)\.r\_\{\\theta\}\(x\)=3\\ Warm\(x\)\+0\.3\\ R\(x\),\\ r\_\{\\phi\}\(x\)=3\\ Cool\(x\)\+0\.3\\ R\(x\)\.Let\(μ0,σ0\)∈ℝ3×ℝ3\(\\mu\_\{0\},\\sigma\_\{0\}\)\\in\\mathbb\{R\}^\{3\}\\times\\mathbb\{R\}^\{3\}denote the channel\-wise mean and standard deviation of CIFAR\-10 images\.R​\(x\)R\(x\)is the lightweight regularizer that encourages generated images to match\(μ0,σ0\)\(\\mu\_\{0\},\\sigma\_\{0\}\), andR​\(x\)=−\(∥μ​\(x\)−μ0∥2\+∥σ​\(x\)−σ0∥2\)R\(x\)=\-\\big\(\\lVert\\mu\(x\)\-\\mu\_\{0\}\\rVert\_\{2\}\+\\lVert\\sigma\(x\)\-\\sigma\_\{0\}\\rVert\_\{2\}\\big\)\.

In summary, the rewardsrθ,rϕr\_\{\\theta\},r\_\{\\phi\}respectively bias the generated distribution toward warm/cool hues\. The rewards are used for candidate curation and for measuring preference alignment on the evaluation datasets\.

#### D\.1\.3Additional results of CIFAR\-10 experiments

Additional results of stability and reward experiments\.In[Figure6](https://arxiv.org/html/2605.29267#A4.F6), we show the expected reward meaning values of modelθ\\thetaandϕ\\phi,Jp​\(θt\)J\_\{p\}\(\\theta\_\{t\}\)andJq​\(ϕt\)J\_\{q\}\(\\phi\_\{t\}\)respectively, at different iterations under settingsA​1A1\-A​6A6\. As shown, both metrics stabilize in the later iterations, indicating that the training dynamics approach a convergent regime\. Therefore, letθ∗=θ54,ϕ∗=ϕ54\\theta^\{\*\}=\\theta\_\{54\},\\phi^\{\*\}=\\phi\_\{54\}\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/x5.png)Figure 6:Reward meaning valuesJp​\(θt\)J\_\{p\}\(\\theta\_\{t\}\)andJq​\(ϕt\)J\_\{q\}\(\\phi\_\{t\}\)on the fixed evaluation dataset at different iterations for settingsA​1−A​6A1\-A6\.Additional analysis of model generated images\.In[Figure7](https://arxiv.org/html/2605.29267#A4.F7),[8](https://arxiv.org/html/2605.29267#A4.F8),[9](https://arxiv.org/html/2605.29267#A4.F9), we show 256 images generated by modelθ\\thetaandϕ\\phiat different training iterations, conditioned on the labels from the fixed evaluation datasets, for the baseline as well as theA​1,A​5A1,A5andA​6A6settings\. ForA​1A1andA​5A5, as retraining proceeds, a growing fraction of model generated samples become blurry, near\-uniform color images, for which the original class label is no longer visually discernible\. This behavior is a direct consequence of our experimental design: to induce strong preference conflict, we define a reward function that is easiest to maximize when an image becomes largely single\-colored\. The observation is also consistent with prior findings\(Ferbachet al\.,[2024](https://arxiv.org/html/2605.29267#bib.bib6)\)that the model parameter related to the maximal reward value usually do not coincide with the model optimal parameter under standard training objectives\.

Recall that modelθ\\thetatends to generate warm\-toned images, whereas modelϕ\\phiprefers cool\-toned images\. Under theA​1A1setting, modelθ\\thetais trained only on its own curated data together with real data, while all curated data used to train modelϕ\\phiare sourced from modelθ\\theta\. As a result, images generated by modelϕ\\phigradually shift toward warm tones as training proceeds in[Figure7](https://arxiv.org/html/2605.29267#A4.F7)\. Moreover, increasing the proportion of curated data in modelθ\\theta’s training dataset causes both models’ outputs to become warm\-toned earlier than before, and the effect is more pronounced \(bottom 2 rows in[Figure7](https://arxiv.org/html/2605.29267#A4.F7)\)\. This qualitative shift is consistent with the quantitative trends: modelθ\\theta’s reward increases, while modelϕ\\phi’s reward decreases after improving the human curation strength of modelθ\\thetain[Figure4](https://arxiv.org/html/2605.29267#S5.F4)\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/figures/image_A1_cu5cu6.jpg)Figure 7:Samples generated by modelθ\\thetaandϕ\\phiwith different curation and cross\-model data proportions\. \(1\) Top: Baseline model output results after each update for 5 iteration\. \(2\) Middle 2 rows: The output results of modelθ\\thetaandϕ\\phiafter different rounds of iterative training using 50% curation data and 50% real data under settingA​1A1\(λθϕ=0,λϕθ=1\\lambda\_\{\\theta\}^\{\\phi\}=0,\\lambda\_\{\\phi\}^\{\\theta\}=1\)\. \(3\) Bottom 2 rows: The output results of modelθ\\thetaandϕ\\phiunder settingA​1A1\. Modelθ\\thetauses 60% curation data and 40% real data for iterative training, while modelϕ\\phiuses 50% curation data and 50% real data\.Under theA​5A5setting, only a small fraction of curation data for each model is cross\-sourced\. The majority of curated samples are still self\-generated\. Compare toA​1A1, modelθ\\theta’s generated images are therefore less strongly warm\-toned, and modelϕ\\phi’s generated images retain more cool\-toned characteristics \([Figure8](https://arxiv.org/html/2605.29267#A4.F8)\)\. Moreover, increasing modelθ\\theta’s curation strength \(λℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\) to 0\.6 inA​5A5makes more warm\-toned samples to be selected and reused for training\. As a result, relative to theA​5A5setting with smallerλℋθ=0\.5\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.5, modelθ\\theta’s generations shift moderately toward warmer tones, while the cool\-toned appearance of modelϕ\\phi’s outputs is substantially attenuated \(bottom 2 rows in[Figure8](https://arxiv.org/html/2605.29267#A4.F8)\)\.

ForA​6A6, the model generated images in[Figure9](https://arxiv.org/html/2605.29267#A4.F9)after convergence at iteration 54 are not similar to the images generated under other settings, which contain images with strong warm or cool tones approaching pure colors, and the final rewardsJp​\(θ∗\),Jq​\(ϕ∗\)J\_\{p\}\(\\theta^\{\*\}\),J\_\{q\}\(\\phi^\{\*\}\)are relatively small\. Due to the strong coupling between the two models \(λθϕ=0\.7,λϕθ=0\.8\\lambda\_\{\\theta\}^\{\\phi\}=0\.7,\\lambda\_\{\\phi\}^\{\\theta\}=0\.8\), taking modelθ\\thetaas an example, its curation samples are largely generated by modelϕ\\phi\. Therefore, if modelθ\\thetais trained to generate high\-reward samples \(warm\-toned images\) through human curation, it requires that modelϕ\\phican consistently produce a sufficient number of warm\-toned images\. However, modelϕ\\phiis also inversely affected by the images generated from modelθ\\thetain its own curation\. After the warm\-toned samples curated by modelθ\\thetaare used to train modelϕ\\phi, modelϕ\\phitends to generate warm\-toned images, but these images are low\-scored byrϕr\_\{\\phi\}\. Therefore, by Eq\. \([3](https://arxiv.org/html/2605.29267#S2.E3)\), the curation selection is almost random, and due to the relatively high ratios of real data \(λℛθ,λℛϕ≥0\.4\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\\geq 0\.4\), warm\-toned images cannot be stably generated under the model interactions which leading the generated samples’ hue shiftsnon\-monotonically and oscillatesduring the iterative process\. After convergence, the system tends togenerate images with tones closer to real datain[Figure9](https://arxiv.org/html/2605.29267#A4.F9), resulting in lowJp​\(θ∗\),Jq​\(ϕ∗\)J\_\{p\}\(\\theta^\{\*\}\),J\_\{q\}\(\\phi^\{\*\}\)values\. In contrast, forA​1A1\-A​5A5, there exists a stable output of curation samples, allowing both models to converge to generating strongly warm/cool\-toned images\.

Moreover, under theA​6A6setting, whenλℋθ=0\.5\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=0\.5, both modelsθ\\thetaandϕ\\phiin the early stage of iterations tend to generate cool\-toned images in[Figure9](https://arxiv.org/html/2605.29267#A4.F9)\(Top 2 rows\)\. However, after improvingλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}to 0\.6, since the curation strength of modelθ\\thetais stronger, the models tend to generate warm\-toned images in the early stage of iterations shown in[Figure9](https://arxiv.org/html/2605.29267#A4.F9)\(Bottom 2 rows\)\.

![Refer to caption](https://arxiv.org/html/2605.29267v1/figures/image_A5_cu5cu6.jpg)Figure 8:Samples generated by modelθ\\thetaandϕ\\phiwith different curation and cross\-model data proportions\. \(1\) Top 2 rows: The output results of modelθ\\thetaandϕ\\phiafter different rounds of iterative training using 50% curation data and 50% real data under settingA​5A5\(λθϕ=0\.2,λϕθ=0\.3\\lambda\_\{\\theta\}^\{\\phi\}=0\.2,\\lambda\_\{\\phi\}^\{\\theta\}=0\.3\)\. \(2\) Bottom 2 rows: The output results of modelθ\\thetaandϕ\\phiunder settingA​5A5\. Modelθ\\thetauses 60% curation data and 40% real data for iterative training, while modelϕ\\phiuses 50% curation data and 50% real data\.![Refer to caption](https://arxiv.org/html/2605.29267v1/figures/image_A6_cu5cu6.jpg)Figure 9:Samples generated by modelθ\\thetaandϕ\\phiwith different curation and cross\-model data proportions\. \(1\) Top 2 rows: The output results of modelθ\\thetaandϕ\\phiafter different rounds of iterative training using 50% curation data and 50% real data under settingA​6A6\(λθϕ=0\.7,λϕθ=0\.8\\lambda\_\{\\theta\}^\{\\phi\}=0\.7,\\lambda\_\{\\phi\}^\{\\theta\}=0\.8\)\. \(2\) Bottom 2 rows: The output results of modelθ\\thetaandϕ\\phiunder settingA​6A6\. Modelθ\\thetauses 60% curation data and 40% real data for iterative training, while modelϕ\\phiuses 50% curation data and 50% real data\.

### D\.2Preference domain mismatch and Qwen2\.5\-0\.5B experiment settings

In this section, we conduct experiments with language models and observe thatpreference domain mismatchcan effectively mask coupling effects\.

Preference domain mismatch\.In our earlier image\-model experiments, modelsθ\\thetaandϕ\\phiare trained based on the same dataset CIFAR\-10, and both models’ preferences are tied to image tone\. In that setting, conflicting preferences across models directly influence whether the images generated at evaluation stage aligned with human preferences\. In practice, however, cross\-model data reuse typically requires only that thedata modalitymatches, without requiring that thepreference data distributionsto be perfectly identical\. For example, in supervised learning, modelθ\\thetamay be trained to generate brightly colored images of flowers due to human preference and curation, and these generated images may enter a large\-scale web corpus\. If modelϕ\\phiinstead is trained to generating cool\-toned images, it may still scrape and reuse modelθ\\theta’s images for training, since they are of the correct modality\. Although the underlying preferences \(bright vs\. cool tones\) are strongly conflicting, this coupling does not necessarily influence modelϕ\\phi’s rewards \(Jq​\(ϕt\)J\_\{q\}\(\\phi\_\{t\}\)\) a lot if modelϕ\\phi’s downstream use case does not require generating flower images\. We refer to this phenomenon aspreference domain mismatch, where strongly conflicting preferences can be present in the training data yet have negligible impact on the evaluated reward due to a mismatch between the preference domain and the evaluation/task domain\. In summary,preference domain mismatchrepresents a situation where preference conflicts exist and are coupled, but are not visible to evaluation rewardsJp,JqJ\_\{p\},J\_\{q\}\.

The relationship between preference domain mismatch and our theory\.This observation is fully consistent with our theory\. The intuitive definition of preference domain mismatch in multi\-model self\-consuming ecosystem is that the data variations driven by the preferences of modelθ\\thetamainly occur within a certain ”content domain/task domain”, but the evaluation scenario of modelϕ\\phidoes not depend on this domain\. We consider the extreme case where all the curation data in the training set of modelθ\\thetaare generated by modelϕ\\phi, which has a preference domain mismatch\. Letδ​θ∗=∂θ∗∂λℋθ\\delta\\theta^\{\*\}=\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}for simplicity\. For modelθ\\theta, its curated samples \(ℋtθ\\mathcal\{H\}^\{\\theta\}\_\{t\}\) concentrate on regions or feature dimensions that are poorly represented in the evaluation datasetℰθ\\mathcal\{E\}\_\{\\theta\}, so their induced model parameter updates have limited local sensitivity on the evaluation log\-likelihood geometry\. Approximately, we assume that

𝔼y∼ℰθ,x∼pθ∗​\(x\|y\)​\[∇θlog⁡pθ∗​\(x\|y\)T\]​δ​θ∗=𝔼y∼ℰθ,x∼pθ∗​\(x\|y\)​\[∇θlog⁡pθ∗​\(x\|y\)T​δ​θ∗\]≈0\.\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta^\{\*\}\}\(x\|y\)\}\\left\[\\nabla\_\{\\theta\}\\log p\_\{\\theta^\{\*\}\}\(x\|y\)^\{T\}\\right\]\\delta\\theta^\{\*\}=\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta^\{\*\}\}\(x\|y\)\}\\left\[\\nabla\_\{\\theta\}\\log p\_\{\\theta^\{\*\}\}\(x\|y\)^\{T\}\\delta\\theta^\{\*\}\\right\]\\approx 0\.\(12\)Denote the Fisher matrix as

Fθ≜𝔼y∼ℰθ,x∼pθ∗​\(x\|y\)​\[∇θlog⁡pθ∗​\(x\|y\)T​∇θlog⁡pθ∗​\(x\|y\)\]\.F\_\{\\theta\}\\triangleq\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta^\{\*\}\}\(x\|y\)\}\\left\[\\nabla\_\{\\theta\}\\log p\_\{\\theta^\{\*\}\}\(x\|y\)^\{T\}\\nabla\_\{\\theta\}\\log p\_\{\\theta^\{\*\}\}\(x\|y\)\\right\]\.By Eq\. \([12](https://arxiv.org/html/2605.29267#A4.E12)\),δ​θ∗T​Fθ​δ​θ∗≈0\\delta\{\\theta^\{\*\}\}^\{T\}F\_\{\\theta\}\\delta\\theta^\{\*\}\\approx 0\. Recall that in[Section4](https://arxiv.org/html/2605.29267#S4),∂Jp​\(θ∗\)∂λℋθ=∇θJp​\(θ∗\)​∂θ∗∂λℋθ\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}, and∇θJp​\(θ∗\)=𝔼y∼ℰθ,x∼pθ∗​\(x\|y\)​\[rθ​\(x,y\)​∇θlog⁡pθ∗​\(x\|y\)\]\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)=\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta^\{\*\}\}\(x\|y\)\}\\left\[r\_\{\\theta\}\(x,y\)\\nabla\_\{\\theta\}\\log p\_\{\\theta^\{\*\}\}\(x\|y\)\\right\]\. Therefore, by Cauchy\-Schwarz inequality,

\|∂Jp​\(θ∗\)∂λℋθ\|≤𝔼y∼ℰθ,x∼pθ∗​\(x\|y\)​\[rθ​\(x,y\)2\]​δ​θ∗T​Fθ​δ​θ∗≈0,\\left\|\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\|\\leq\\sqrt\{\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta^\{\*\}\}\(x\|y\)\}\\left\[r\_\{\\theta\}\(x,y\)^\{2\}\\right\]\}\\sqrt\{\\delta\{\\theta^\{\*\}\}^\{T\}F\_\{\\theta\}\\delta\\theta^\{\*\}\}\\approx 0,\(13\)which means∂Jp​\(θ∗\)∂λℋθ≈0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\approx 0\.

Eq\. \([13](https://arxiv.org/html/2605.29267#A4.E13)\) formalizes how coupling can be masked with preference alignment mismatch: even if cross\-model curation data induces a large parameter response \(∥δ​θ∗∥\\lVert\\delta\\theta^\{\*\}\\rVert\), the rewardJpJ\_\{p\}can be only weakly sensitive toλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}whenever the ”visible component” under the evaluation Fisher metric,δ​θ∗T​Fθ​δ​θ∗\\delta\{\\theta^\{\*\}\}^\{T\}F\_\{\\theta\}\\delta\\theta^\{\*\}, is small\. With strong preference domain mismatch, the mean\-squared projection of the parameter response direction onto the score function space,𝔼y∼ℰθ,x∼pθ∗​\(x\|y\)​\[\(δ​θ∗T​∇θlog⁡pθ∗​\(x\|y\)\)2\]=δ​θ∗T​Fθ​δ​θ∗\\mathbb\{E\}\_\{y\\sim\\mathcal\{E\}\_\{\\theta\},x\\sim p\_\{\\theta^\{\*\}\}\(x\|y\)\}\\left\[\\left\(\\delta\{\\theta^\{\*\}\}^\{T\}\\nabla\_\{\\theta\}\\log p\_\{\\theta^\{\*\}\}\(x\|y\)\\right\)^\{2\}\\right\]=\\delta\{\\theta^\{\*\}\}^\{T\}F\_\{\\theta\}\\delta\\theta^\{\*\}, is sufficiently small–equivalently,δ​θ∗\\delta\{\\theta^\{\*\}\}lies mostly in \(or near\) the null space of the Fisher matrix\.

Experiment training setting\.We use the pretrained Qwen2\.5\-0\.5B\-Instruct\(Team,[2024](https://arxiv.org/html/2605.29267#bib.bib65)\)as the shared base model for both modelθ\\thetaandϕ\\phi, with separate LoRA adapters\. Our LoRA configuration follows the standard LoRA formulation\(Huet al\.,[2022](https://arxiv.org/html/2605.29267#bib.bib62)\)with rankr=16,α=32r=16,\\ \\alpha=32,dropout=0\.05\.

For modelθ\\theta, we design it to tend towards summarizing the input text and adopt XSum\(Narayanet al\.,[2018](https://arxiv.org/html/2605.29267#bib.bib64)\)which pairs long news articles with a short single\-sentence summary asℛθ\\mathcal\{R\}^\{\\theta\}\. Assume modelϕ\\phiprefers to paraphrase the input text, andℛϕ\\mathcal\{R\}^\{\\phi\}is the paraphrasing data from CoEdIT\(Rahejaet al\.,[2023](https://arxiv.org/html/2605.29267#bib.bib63)\), where the text data length is significantly shorter compared to the article data in XSum\. The training data size for both models are 1024, and we finetune both models for 13 epochs with training steps are set to be 512\. Models generate synthetic data using temperatureT=0\.9T=0\.9and top\-p=0\.95p=0\.95, and when calculating rewards on evaluation sets,T=0T=0and top\-p=1p=1\. The evaluation datasetsℰθ,ℰϕ\\mathcal\{E\}\_\{\\theta\},\\mathcal\{E\}\_\{\\phi\}for calculatingJp​\(θt\),Jq​\(ϕt\)J\_\{p\}\(\\theta\_\{t\}\),J\_\{q\}\(\\phi\_\{t\}\)are selected from XSum and CoEdIT, respectively, and they are independent of the real dataℛθ,ℛϕ\\mathcal\{R\}^\{\\theta\},\\mathcal\{R\}^\{\\phi\}used for training\. Moreover, we keep fixed system prompts throughout training and evaluation:

- •modelθ\\theta:”You are a helpful English summarization assistant\. Write a single\-sentence, information\-dense summary that is as short as possible\. Do not add new facts or embellishments\.”
- •modelϕ\\phi:”You are an English paraphrasing assistant\. Rewrite the text with different words while maintaining the core meaning\. Do not add new facts\.”

The iterative retraining process is the same as in[Algorithm1](https://arxiv.org/html/2605.29267#alg1)\. For modelθ\\theta, the cross\-model training curation inputs are sentence\-level, while the evaluation inputs for summarization when calculatingJp​\(θt\)J\_\{p\}\(\\theta\_\{t\}\)are long articles, leading to a mismatch between the induced training distribution and the target evaluation domain\. This mismatch is further reflected in the definitions of rewards below\.

Reward setting\.Assumexxis the model input andyyis the model output\. For modelθ\\theta, its summarization rewardrθr\_\{\\theta\}is defined as a weighted combination of a length preference term, a keyword\-coverage term, and a copying penalty\. The length term encourages single\-sentence, information\-dense summaries by scoring the token lengthL​\(y\)L\(y\)throughsl​e​n=exp⁡\(−\|L​\(y\)−12\|2\)s\_\{len\}=\\exp\(\-\\frac\{\|L\(y\)\-12\|\}\{2\}\), where the denominator controls the sharpness of the preference\. The coverage term measures whetheryycontains salient words from the input text, and we take the top\-20 most frequent tokens inxxand compute the fraction that appear inyyassc​o​v∈\[0,1\]s\_\{cov\}\\in\[0,1\]\. To discourage copying, we compute a 4\-gram overlap ratio betweenxxandyyand getsc​o​p​ys\_\{copy\}\.rθ​\(x,y\)=0\.55​sl​e​n\+0\.55​sc​o​v−0\.6​sc​o​p​yr\_\{\\theta\}\(x,y\)=0\.55s\_\{len\}\+0\.55s\_\{cov\}\-0\.6s\_\{copy\}\.

For modelϕ\\phi, to better align with its paraphrasing objective, we definerϕr\_\{\\phi\}as the weighted sum of a length\-ratio constraints^l​e​n\\hat\{s\}\_\{len\}, a formality scores^f​o​r​m\\hat\{s\}\_\{form\}and a moderate copying terms^m​i​d\\hat\{s\}\_\{mid\}that penalizes both near\-verbatim copying and excessive divergence\.s^l​e​n=exp⁡\(−\|L​\(y\)L​\(x\)−1\|0\.25\)\\hat\{s\}\_\{len\}=\\exp\(\-\\frac\{\|\\frac\{L\(y\)\}\{L\(x\)\}\-1\|\}\{0\.25\}\)\.s^f​o​r​m\\hat\{s\}\_\{form\}is defined based on contraction usage, and we count common English contractions \(e\.g\. ”can’t”, ”we’re”\) and tokens ending with ”n’t”\. Fewer contractions correspond to more formal writing and letsf​o​r​m=1−\#​c​o​n​t​r​a​c​t​i​o​n​sL​\(y\)s\_\{form\}=1\-\\frac\{\\\#contractions\}\{L\(y\)\}\. For semantic preservation, we compute the same 4\-gram copy ratiosc​o​p​ys\_\{copy\}as modelθ\\thetaand score it with a kernel centered at 0\.2,s^m​i​d=exp⁡\(−\|sc​o​p​y−0\.2\|0\.1\)\\hat\{s\}\_\{mid\}=\\exp\(\-\\frac\{\|s\_\{copy\}\-0\.2\|\}\{0\.1\}\)\. The paraphrasing rewardrϕ​\(x,y\)=0\.45​sl​e​n\+0\.35​sf​o​r​m\+0\.3​sm​i​dr\_\{\\phi\}\(x,y\)=0\.45s\_\{len\}\+0\.35s\_\{form\}\+0\.3s\_\{mid\}\.

### D\.3Mechanism validation experiment setting

There are two factors making computing mechanism results in Section[4](https://arxiv.org/html/2605.29267#S4)on large models difficult: sampling error and statistical error from the approximation methods to estimate high\-dimensional matrices\. Since theory\-related matrices are local and involve second\-order information, this is closely related to the well\-known difficulty of estimating inverse\-Hessian\-based quantities in large models\. Prior works treat scalable and statistically stable estimation of such objects as a nontrivial problem\(Liet al\.,[2025](https://arxiv.org/html/2605.29267#bib.bib16); Basuet al\.,[2021](https://arxiv.org/html/2605.29267#bib.bib17)\)\. Therefore, to validate the mechanism more detailed, we extend Example[4\.6](https://arxiv.org/html/2605.29267#S4.Thmtheorem6)in Section[5](https://arxiv.org/html/2605.29267#S5)\.

Recall that the mixture distributionsP=\(1−λℋθ\)​𝒩​\(ϕ,σ2​I\)\+λℋθ​𝒩​\(ϕ\+a,σ2​I\)P=\(1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\\mathcal\{N\}\(\\phi,\\sigma^\{2\}I\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\\mathcal\{N\}\(\\phi\+a,\\sigma^\{2\}I\)andQ=𝒩​\(A​\(t\)​θ,σ2​I\)Q=\\mathcal\{N\}\(A\(t\)\\theta,\\sigma^\{2\}I\)whereA​\(t\)∈ℝ24×24A\(t\)\\in\\mathbb\{R\}^\{24\\times 24\}is a block\-diagonal matrix with 122×22\\times 2blocks\. Theii\-th blockAi​\(t\)=t​βi​RiA\_\{i\}\(t\)=t\\beta\_\{i\}R\_\{i\}wheret∈\[0\.05,1\]t\\in\[0\.05,1\]is the coupling scale,βi∈\[0\.08,0\.95\]\\beta\_\{i\}\\in\[0\.08,0\.95\]controls the interaction magnitude of blockii, and

Ri=\(cos⁡xi,−sin⁡xisin⁡xi,cos⁡xi\),\{xi\}=\{−80∘,−66∘,−52∘,−38∘,−24∘,−10∘,5∘,21∘,37∘,53∘,69∘,85∘\}R\_\{i\}=\\begin\{pmatrix\}\\cos x\_\{i\},\-\\sin x\_\{i\}\\\\ \\sin x\_\{i\},\\cos x\_\{i\}\\end\{pmatrix\},\\\{x\_\{i\}\\\}=\\\{\-80^\{\\circ\},\-66^\{\\circ\},\-52^\{\\circ\},\-38^\{\\circ\},\-24^\{\\circ\},\-10^\{\\circ\},5^\{\\circ\},21^\{\\circ\},37^\{\\circ\},53^\{\\circ\},69^\{\\circ\},85^\{\\circ\}\\\}is a 2\-dimensional matrix introduces local geometric heterogeneity\.a=\[a1,…,a12\]∈ℝ24a=\[a\_\{1\},\.\.\.,a\_\{12\}\]\\in\\mathbb\{R\}^\{24\}andai=0\.9i​\(1\+0\.2​sin⁡2​π​\(i−1\)11\)​\(cos\(140​\(i−1\)11−70\)∘,−sin\(140​\(i−1\)11−70\)∘sin\(140​\(i−1\)11−70\)∘,cos\(140​\(i−1\)11−70\)∘\)​\(1,2\)Ta\_\{i\}=0\.9^\{i\}\\big\(1\+0\.2\\sin\\frac\{2\\pi\(i\-1\)\}\{11\}\\big\)\\begin\{pmatrix\}\\cos\(\\frac\{140\(i\-1\)\}\{11\}\-70\)^\{\\circ\},\-\\sin\(\\frac\{140\(i\-1\)\}\{11\}\-70\)^\{\\circ\}\\\\ \\sin\(\\frac\{140\(i\-1\)\}\{11\}\-70\)^\{\\circ\},\\cos\(\\frac\{140\(i\-1\)\}\{11\}\-70\)^\{\\circ\}\\end\{pmatrix\}\(1,2\)^\{T\}\. We set the reward expectationsJp​\(θ\)=gpT​θ−ηp​∥θ∥22J\_\{p\}\(\\theta\)=g\_\{p\}^\{T\}\\theta\-\\frac\{\\eta\_\{p\}\\lVert\\theta\\rVert^\{2\}\}\{2\}withηp=0\.18\\eta\_\{p\}=0\.18,Jq​\(ϕ\)=gqT​ϕ−ηq​∥ϕ∥22J\_\{q\}\(\\phi\)=g\_\{q\}^\{T\}\\phi\-\\frac\{\\eta\_\{q\}\\lVert\\phi\\rVert^\{2\}\}\{2\}withηq=0\.22\\eta\_\{q\}=0\.22,gp=\(gp​1,…,gp​12\)∈ℝ24g\_\{p\}=\(g\_\{p1\},\.\.\.,g\_\{p12\}\)\\in\\mathbb\{R\}^\{24\}andgp​i=0\.9i​\(1\+0\.15​cos⁡1\.5​π​\(i−1\)11\)​\(cos\(80​\(i−1\)11−55\)∘,−sin\(80​\(i−1\)11−55\)∘sin\(80​\(i−1\)11−55\)∘,cos\(80​\(i−1\)11−55\)∘\)​\(1,0\)Tg\_\{pi\}=0\.9^\{i\}\\big\(1\+0\.15\\cos\\frac\{1\.5\\pi\(i\-1\)\}\{11\}\\big\)\\begin\{pmatrix\}\\cos\(\\frac\{80\(i\-1\)\}\{11\}\-55\)^\{\\circ\},\-\\sin\(\\frac\{80\(i\-1\)\}\{11\}\-55\)^\{\\circ\}\\\\ \\sin\(\\frac\{80\(i\-1\)\}\{11\}\-55\)^\{\\circ\},\\cos\(\\frac\{80\(i\-1\)\}\{11\}\-55\)^\{\\circ\}\\end\{pmatrix\}\(1,0\)^\{T\},gq=\(gq​1,…,gq​12\)∈ℝ24g\_\{q\}=\(g\_\{q1\},\.\.\.,g\_\{q12\}\)\\in\\mathbb\{R\}^\{24\}andgq​i=0\.9i​\(1\+0\.18​sin⁡\(1\.5​π​\(i−1\)11\+0\.3\)\)​\(cos\(100​\(i−1\)11−40\)∘,−sin\(100​\(i−1\)11−40\)∘sin\(100​\(i−1\)11−40\)∘,cos\(100​\(i−1\)11−40\)∘\)​\(1,4\)Tg\_\{qi\}=0\.9^\{i\}\\big\(1\+0\.18\\sin\(\\frac\{1\.5\\pi\(i\-1\)\}\{11\}\+0\.3\)\\big\)\\begin\{pmatrix\}\\cos\(\\frac\{100\(i\-1\)\}\{11\}\-40\)^\{\\circ\},\-\\sin\(\\frac\{100\(i\-1\)\}\{11\}\-40\)^\{\\circ\}\\\\ \\sin\(\\frac\{100\(i\-1\)\}\{11\}\-40\)^\{\\circ\},\\cos\(\\frac\{100\(i\-1\)\}\{11\}\-40\)^\{\\circ\}\\end\{pmatrix\}\(1,4\)^\{T\}\. For simplicity, since the total iteration number is 100, we use the mean value of model parameters after the 80th iteration as the approximation forθ∗\\theta^\{\*\}andϕ∗\\phi^\{\*\}\. Moreover, we build the system includinga,gp,gq,Ra,g\_\{p\},g\_\{q\},Rby repeating a common two dimensional coupling template across blocks\.RiR\_\{i\}is set to be an orthogonal rotation, whileai,gp​i,gq​ia\_\{i\},g\_\{pi\},g\_\{qi\}are matched blockwise curation and reward directions\. This yields a multi\-mode coupled system that remains analytically tractable, cleanly separates coupling magnitude from geometric distortion\.

Based on the blockwise Gaussian settings, it’s easy to check that the self\-influence⟨∇θJp​\(θ∗\),Sp​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩=∑i=112⟨∇θiJp​\(θ∗\),Sp,i​𝔼PH​\[−∇θiℓθ​\(θ∗\)\]⟩\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle=\\sum\_\{i=1\}^\{12\}\\langle\\nabla\_\{\\theta\_\{i\}\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p,i\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\_\{i\}\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\ranglewhereθi\\theta\_\{i\},Sp,iS\_\{p,i\}are the components ofθ\\thetaandSpS\_\{p\}within specific dimension blocks, respectively, andθ=\(θ1,…,θ12\)\\theta=\(\\theta\_\{1\},\.\.\.,\\theta\_\{12\}\),Sp=d​i​a​g​\(Sp,1,…,Sp,12\)S\_\{p\}=diag\(S\_\{p,1\},\.\.\.,S\_\{p,12\}\),∇θJp​\(θ∗\)=\(∇θ1Jp​\(θ∗\),…,∇θ12Jp​\(θ∗\)\)\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)=\\big\(\\nabla\_\{\\theta\_\{1\}\}J\_\{p\}\(\\theta^\{\*\}\),\.\.\.,\\nabla\_\{\\theta\_\{12\}\}J\_\{p\}\(\\theta^\{\*\}\)\\big\)and𝔼PH​\[−∇θℓθ​\(θ∗\)\]=\(𝔼PH​\[−∇θ1ℓθ​\(θ∗\)\],…,𝔼PH​\[−∇θ12ℓθ​\(θ∗\)\]\)\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]=\\big\(\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\_\{1\}\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\],\.\.\.,\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\_\{12\}\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\big\)\. Similarly,

⟨∇θJp​\(θ∗\),𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩=∑i=112⟨∇θiJp​\(θ∗\),𝔼PH​\[−∇θiℓθ​\(θ∗\)\]⟩,\\displaystyle\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle=\\sum\_\{i=1\}^\{12\}\\langle\\nabla\_\{\\theta\_\{i\}\}J\_\{p\}\(\\theta^\{\*\}\),\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\_\{i\}\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle,⟨∇ϕJq​\(ϕ∗\),Cq​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩=∑i=112⟨∇ϕiJq​\(ϕ∗\),Cq,i​𝔼PH​\[−∇θiℓθ​\(θ∗\)\]⟩,\\displaystyle\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),C\_\{q\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle=\\sum\_\{i=1\}^\{12\}\\langle\\nabla\_\{\\phi\_\{i\}\}J\_\{q\}\(\\phi^\{\*\}\),C\_\{q,i\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\_\{i\}\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle,⟨∇ϕJq​\(ϕ∗\),Sq​Cq​𝔼PH​\[−∇θℓθ​\(θ∗\)\]⟩=∑i=112⟨∇ϕiJq​\(ϕ∗\),Sq,i​Cq,i​𝔼PH​\[−∇θiℓθ​\(θ∗\)\]⟩\.\\displaystyle\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),S\_\{q\}C\_\{q\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle=\\sum\_\{i=1\}^\{12\}\\langle\\nabla\_\{\\phi\_\{i\}\}J\_\{q\}\(\\phi^\{\*\}\),S\_\{q,i\}C\_\{q,i\}\\mathbb\{E\}\_\{P\_\{H\}\}\[\-\\nabla\_\{\\theta\_\{i\}\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle\.In Figure[3](https://arxiv.org/html/2605.29267#S5.F3)\(E\)\(F\), the value corresponding to theii\-th dimension block on the horizontal axis corresponds to theii\-th term in the above equations\.

## Appendix ETheory Extension And Generalization

First, in the main paper, our theory is based on the assumption that the outputs of models are each other’s inputs,x=pθ​\(x\|y\)x=p\_\{\\theta\}\(x\|y\)andy=qϕ​\(y\|x\)y=q\_\{\\phi\}\(y\|x\)\. It’s easy to verify that our theory can be extended to simpler unsupervised learning multi\-model systems or systems where the input and output of both models are the same,x=pθ​\(x\|y\)x=p\_\{\\theta\}\(x\|y\)andx=qϕ​\(x\|y\)x=q\_\{\\phi\}\(x\|y\)\. Next, we discuss extensions of some theory results presented in the main paper\.

### E\.1Alignment improvement in the single\-model scenario

###### Corollary E\.1\.

For a single modelθ\\theta, under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5), ifρp≈1\\rho\_\{p\}\\approx 1, then∂Jp​\(θ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0\.

### E\.2Sufficient conditions related to cross\-model influence

###### Corollary E\.2\.

Denoteτq=γϕ−Lϕ​εϕ−Lθ​εθ​Lϕ​εϕγθ−Lθ​εθ\\tau\_\{q\}=\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}and letmqm\_\{q\}be the minimal eigenvalue ofSq\+SqT2\\frac\{S\_\{q\}\+S\_\{q\}^\{T\}\}\{2\}, we have∥Sq∥≤1τq\\lVert S\_\{q\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{q\}\}and if\|ρq\|\>11\+mq2​τq2\|\\rho\_\{q\}\|\>\\frac\{1\}\{\\sqrt\{1\+m\_\{q\}^\{2\}\\tau\_\{q\}^\{2\}\}\}, thensign⁡\(ρq\)⋅∂Jq​\(β∗\)∂λℋθ<0\\operatorname\{sign\}\(\\rho\_\{q\}\)\\cdot\\frac\{\\partial J\_\{q\}\(\\beta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}<0\.

### E\.3The extension of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)

For the curation ratioλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}, changing the size of real dataset, synthetic dataset, or curated synthetic dataset will influenceλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\. Different combinations of such modifications lead to different forms of equations in Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)\. For instance, varying the number of real data samples changesλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}, but the expression in Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)contains no term explicitly associated with real datasetℛθ\\mathcal\{R\}^\{\\theta\}\. Mathematically, if we regardJpJ\_\{p\}as a multivariate function ofλℋθ,λℛθ,λ𝒮θ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}subject to the constraintλℋθ\+λℛθ\+λ𝒮θ=1\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}=1, then each data\-modification scheme corresponds to a particular definition of the total derivative∂Jp∂λℋθ\\frac\{\\partial J\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\.

Generally, we suppose the variation inλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}in practice is induced by adjusting the dataset sizes ofℛθ,𝒮θ,ℋθ\\mathcal\{R\}^\{\\theta\},\\mathcal\{S\}^\{\\theta\},\\mathcal\{H\}^\{\\theta\}in fixed proportionsar,as,aha\_\{r\},a\_\{s\},a\_\{h\}andar\+as\+ah=1a\_\{r\}\+a\_\{s\}\+a\_\{h\}=1\. For example, addingnntraining samples means addingar​na\_\{r\}nreal data samples,as​na\_\{s\}nsynthetic data samples andah​na\_\{h\}ncurated synthetic samples\. If any ofar,as,aha\_\{r\},a\_\{s\},a\_\{h\}is negative, the corresponding operation represents removing samples rather than adding them\.Note that the settingar=0,as=0,ah=1a\_\{r\}=0,a\_\{s\}=0,a\_\{h\}=1is equivalent to the scenario considered in the main paper\.By varying\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\), this parameterization covers all possible cases\.

Under this parameterization, we can get the following theorem, which extends and generalized Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)\. Notably, theSp,Sq,CqS\_\{p\},S\_\{q\},C\_\{q\}matrices remain unchanged, highlighting the generality of our methods\.

###### Theorem E\.3\.

\[Generalization of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)\] Under the parameterization of\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\)and Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6), the self\-consuming multi\-model system converges\. LetPℋ,P𝒮P\_\{\\mathcal\{H\}\},P\_\{\\mathcal\{S\}\}be the distributions of human\-curation dataℋtθ\\mathcal\{H\}\_\{t\}^\{\\theta\}and synthetic self\-consuming data𝒮tθ\\mathcal\{S\}\_\{t\}^\{\\theta\}after convergence, respectively, and we have

∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=1ah−λℋθ​⟨∇θJp​\(θ∗\),Sp​\[as​𝔼z∼P𝒮​∇θℓθ​\(θ∗\)\+ah​𝔼z∼Pℋ​∇θℓθ​\(θ∗\)\+ar​𝔼z∼ℛθ​∇θℓθ​\(θ∗\)\]⟩,\\displaystyle=\\frac\{1\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\bigg\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\theta\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\Big\]\\bigg\\rangle,∂Jq​\(ϕ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=1ah−λℋθ​⟨∇ϕJq​\(ϕ∗\),Sq​Cq​\[as​𝔼z∼P𝒮​∇θℓθ​\(θ∗\)\+ah​𝔼z∼Pℋ​∇θℓθ​\(θ∗\)\+ar​𝔼z∼ℛθ​∇θℓθ​\(θ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\bigg\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\\,S\_\{q\}C\_\{q\}\\,\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\theta\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\Big\]\\bigg\\rangle\.

###### Proof\.

In this proof, we adopt the same notation as in the proof of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)for simplicity\. See Appendix[F\.4](https://arxiv.org/html/2605.29267#A6.SS4)for details\. The overall proof strategy mirrors that of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5), with the sole difference is how to computed​fθd​λℋθ\\frac\{df\_\{\\theta\}\}\{d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}wherefθf\_\{\\theta\}is the density function of modelθ\\theta’s mixture distributionPP\. Notice that the total derivative offθf\_\{\\theta\}is

d​fθ​\(λ𝒮θ,λℋθ,λℛθ;z,θ,ϕ\)=fθ,s​\(z;θ,ϕ\)​d​λ𝒮θ\+fθ,c​\(z;θ,ϕ\)​d​λℋθ\+fθ,r​\(z\)​d​λℛθ,d​λ𝒮θ\+d​λℋθ\+d​λℛθ=0\.df\_\{\\theta\}\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\};z,\\theta,\\phi\)=f\_\{\\theta,s\}\(z;\\theta,\\phi\)d\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+f\_\{\\theta,c\}\(z;\\theta,\\phi\)d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+f\_\{\\theta,r\}\(z\)d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\ d\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=0\.Under the parameterization of\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\), it’s easy to check thatd​λℛθ=ar−λℛθah−λℋθ​d​λℋθd\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=\\frac\{a\_\{r\}\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}andd​λ𝒮θ=as−λ𝒮θah−λℋθ​d​λℋθd\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}=\\frac\{a\_\{s\}\-\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\. These two equations align with the resultsd​λ𝒮θ=−λ𝒮θλ𝒮θ\+λℛθ​d​λℋθ,d​λℛθ=−λℛθλ𝒮θ\+λℛθ​d​λℋθd\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}=\-\\frac\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\ d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=\-\\frac\{\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}in the proof of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)ifar=as=0a\_\{r\}=a\_\{s\}=0andah=1a\_\{h\}=1\. Therefore, similar to Eq\. \([F\.4](https://arxiv.org/html/2605.29267#A6.Ex75)\),

∂Fp∂λℋθ\\displaystyle\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=∫∇θℓθ​\(z;θ∗\)​d​fθ​\(λ𝒮θ,λℋθ,λℛθ;z,θ∗,ϕ∗\)d​λℋθ​𝑑z\\displaystyle=\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\frac\{df\_\{\\theta\}\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\};z,\\theta^\{\*\},\\phi^\{\*\}\)\}\{d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}dz=∫∇θℓθ​\(z;θ∗\)​\(as−λ𝒮θah−λℋθ​fθ,s​\(z;θ∗,ϕ∗\)\+fθ,c​\(z;θ∗,ϕ∗\)\+ar−λℛθah−λℋθ​fθ,r​\(z\)\)​𝑑z\\displaystyle=\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\big\(\\frac\{a\_\{s\}\-\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}f\_\{\\theta,s\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\frac\{a\_\{r\}\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}f\_\{\\theta,r\}\(z\)\\big\)dz=∫∇θℓθ\(z;θ∗\)\(asah−λℋθfθ,s\(z;θ∗,ϕ∗\)\+ahah−λℋθfθ,c\(z;θ∗,ϕ∗\)\+arah−λℋθfθ,r\(z\)\\displaystyle=\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\big\(\\frac\{a\_\{s\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}f\_\{\\theta,s\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\frac\{a\_\{h\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\frac\{a\_\{r\}\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}f\_\{\\theta,r\}\(z\)−λ𝒮θ​fθ,s​\(z;θ∗,ϕ∗\)\+λℋθ​fθ,c​\(z;θ∗,ϕ∗\)\+λℛθ​fθ,r​\(z\)ah−λℋθ\)dz\\displaystyle\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \-\\frac\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}f\_\{\\theta,s\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}f\_\{\\theta,r\}\(z\)\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\big\)dz=1ah−λℋθ​\[as​𝔼z∼P𝒮​∇θℓθ​\(θ∗\)\+ah​𝔼z∼Pℋ​∇θℓθ​\(θ∗\)\+ar​𝔼z∼ℛθ​∇θℓθ​\(θ∗\)\]\.\\displaystyle=\\frac\{1\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\theta\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\Big\]\.\(14\)Therefore, similar to the left parts in the proof of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5), we can prove this theorem\. ∎

Symmetrically, if we vary only the curation strength of modelϕ\\phi\(λℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\), the following theorem characterizes both its local self\-influence and cross\-influence on the final rewards\.

###### Theorem E\.4\.

\[Symmetrical version ofλℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\] Under the parameterization of\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\)and Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6), the self\-consuming multi\-model system converges\. LetQℋ,Q𝒮Q\_\{\\mathcal\{H\}\},Q\_\{\\mathcal\{S\}\}be the distributions of human\-curation dataℋtϕ\\mathcal\{H\}\_\{t\}^\{\\phi\}and synthetic self\-consuming data𝒮tϕ\\mathcal\{S\}\_\{t\}^\{\\phi\}after convergence, respectively, and we have

∂Jq​\(ϕ∗\)∂λℋϕ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\}=1ah−λℋϕ​⟨∇ϕJq​\(ϕ∗\),Sq​\[as​𝔼z∼Q𝒮​∇ϕℓϕ​\(ϕ∗\)\+ah​𝔼z∼Qℋ​∇ϕℓϕ​\(ϕ∗\)\+ar​𝔼z∼ℛϕ​∇ϕℓϕ​\(ϕ∗\)\]⟩,\\displaystyle=\\frac\{1\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\}\\,\\bigg\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\\,S\_\{q\}\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\phi\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\\Big\]\\bigg\\rangle,∂Jp​\(θ∗\)∂λℋϕ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\}=1ah−λℋϕ​⟨∇θJp​\(θ∗\),Sp​Cp​\[as​𝔼z∼Q𝒮​∇ϕℓϕ​\(ϕ∗\)\+ah​𝔼z∼Qℋ​∇ϕℓϕ​\(ϕ∗\)\+ar​𝔼z∼ℛϕ​∇ϕℓϕ​\(ϕ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{a\_\{h\}\-\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\}\\,\\bigg\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}C\_\{p\}\\,\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\phi\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\\Big\]\\bigg\\rangle\.

In fact, the differences among these data\-modification schemes in their impact on\{∂Ji∂λℋj,i∈\{θ,ϕ\},j∈\{θ,ϕ\}\}\\big\\\{\\frac\{\\partial J\_\{i\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{j\}\},i\\in\\\{\\theta,\\phi\\\},\\ j\\in\\\{\\theta,\\phi\\\}\\big\\\}are attributable to how they compose model parameter update directions induced by different datasetsℛj,𝒮j,ℋj,j∈\{θ,ϕ\}\\mathcal\{R\}^\{j\},\\mathcal\{S\}^\{j\},\\mathcal\{H\}^\{j\},j\\in\\\{\\theta,\\phi\\\}\. Under each scheme, the resulting update is a combination of the per\-dataset update directions, with weights given exactly by the parameterization\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\)\.

Extend to any mixing weight\.Prior results focus on the curation proportionλℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}, as our analysis targets how human curation influences preference alignments\. More generally, our framework can be extended to any mixture weight\. Specifically, under the same parametrization\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\), we have the following theorem\.

###### Theorem E\.5\.

\[Extension to any mixture weight\] Under the parameterization of\(ar,as,ah\)\(a\_\{r\},a\_\{s\},a\_\{h\}\)and Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6), the self\-consuming multi\-model system converges\. LetPℋ,P𝒮P\_\{\\mathcal\{H\}\},P\_\{\\mathcal\{S\}\}be the distributions of human\-curation dataℋtθ\\mathcal\{H\}\_\{t\}^\{\\theta\}and synthetic self\-consuming data𝒮tθ\\mathcal\{S\}\_\{t\}^\{\\theta\}after convergence, respectively\. LetQℋ,Q𝒮Q\_\{\\mathcal\{H\}\},Q\_\{\\mathcal\{S\}\}be the distributions of human\-curation dataℋtϕ\\mathcal\{H\}\_\{t\}^\{\\phi\}and synthetic self\-consuming data𝒮tϕ\\mathcal\{S\}\_\{t\}^\{\\phi\}after convergence, respectively\. If only changing the mixture weights of modelθ\\theta, then for any\(λ,a\)∈\{\(λ𝒮θ,as\),\(λℛθ,ar\)\}\(\\lambda,a\)\\in\\\{\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},a\_\{s\}\),\(\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},a\_\{r\}\)\\\},

∂Jp​\(θ∗\)∂λ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\}=1a−λ​⟨∇θJp​\(θ∗\),Sp​\[as​𝔼z∼P𝒮​∇θℓθ​\(θ∗\)\+ah​𝔼z∼Pℋ​∇θℓθ​\(θ∗\)\+ar​𝔼z∼ℛθ​∇θℓθ​\(θ∗\)\]⟩,\\displaystyle=\\frac\{1\}\{a\-\\lambda\}\\,\\bigg\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\theta\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\Big\]\\bigg\\rangle,∂Jq​\(ϕ∗\)∂λ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\}=1a−λ​⟨∇ϕJq​\(ϕ∗\),Sq​Cq​\[as​𝔼z∼P𝒮​∇θℓθ​\(θ∗\)\+ah​𝔼z∼Pℋ​∇θℓθ​\(θ∗\)\+ar​𝔼z∼ℛθ​∇θℓθ​\(θ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{a\-\\lambda\}\\,\\bigg\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\\,S\_\{q\}C\_\{q\}\\,\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\theta\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\Big\]\\bigg\\rangle\.If only changing the mixture weights of modelϕ\\phi, then for any\(λ,a\)∈\{\(λ𝒮ϕ,as\),\(λℛϕ,ar\)\}\(\\lambda,a\)\\in\\\{\(\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\},a\_\{s\}\),\(\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\},a\_\{r\}\)\\\},

∂Jq​\(ϕ∗\)∂λ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\}=1a−λ​⟨∇ϕJq​\(ϕ∗\),Sq​\[as​𝔼z∼Q𝒮​∇ϕℓϕ​\(ϕ∗\)\+ah​𝔼z∼Qℋ​∇ϕℓϕ​\(ϕ∗\)\+ar​𝔼z∼ℛϕ​∇ϕℓϕ​\(ϕ∗\)\]⟩,\\displaystyle=\\frac\{1\}\{a\-\\lambda\}\\,\\bigg\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\\,S\_\{q\}\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\phi\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\\Big\]\\bigg\\rangle,∂Jp​\(θ∗\)∂λ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\}=1a−λ​⟨∇θJp​\(θ∗\),Sp​Cp​\[as​𝔼z∼Q𝒮​∇ϕℓϕ​\(ϕ∗\)\+ah​𝔼z∼Qℋ​∇ϕℓϕ​\(ϕ∗\)\+ar​𝔼z∼ℛϕ​∇ϕℓϕ​\(ϕ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{a\-\\lambda\}\\,\\bigg\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}C\_\{p\}\\,\\Big\[a\_\{s\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{S\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{h\}\\mathbb\{E\}\_\{z\\sim Q\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\+a\_\{r\}\\mathbb\{E\}\_\{z\\sim\\mathcal\{R\}^\{\\phi\}\}\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(\\phi^\{\*\}\)\\Big\]\\bigg\\rangle\.

Based on Theorem[E\.3](https://arxiv.org/html/2605.29267#A5.Thmtheorem3),[E\.4](https://arxiv.org/html/2605.29267#A5.Thmtheorem4)and[E\.5](https://arxiv.org/html/2605.29267#A5.Thmtheorem5), we can obtain the expressions of the partial derivatives ofJpJ\_\{p\}andJqJ\_\{q\}with respect to different mixing weights\. Therefore, we have the Jacobian ofJpJ\_\{p\}andJqJ\_\{q\}\.

Extend to cross\-model\-generated fractions\.Similarly, the above analysis can be extended to variations in cross\-model proportionsλθϕ\\lambda\_\{\\theta\}^\{\\phi\}andλϕθ\\lambda\_\{\\phi\}^\{\\theta\}\.

### E\.4The extension of Theorem[4\.8](https://arxiv.org/html/2605.29267#S4.Thmtheorem8)

From the proof of[Theorem4\.8](https://arxiv.org/html/2605.29267#S4.Thmtheorem8)in in[SectionF\.9](https://arxiv.org/html/2605.29267#A6.SS9), for any data proportionλj∈\{λ𝒮j,λℋj,λℛj\},j∈\{θ,ϕ\}\\lambda^\{j\}\\in\\\{\\lambda\_\{\\mathcal\{S\}\}^\{j\},\\lambda\_\{\\mathcal\{H\}\}^\{j\},\\lambda\_\{\\mathcal\{R\}\}^\{j\}\\\},j\\in\\\{\\theta,\\phi\\\}and fixed constantsc,d∈\[0,1\]c,d\\in\[0,1\], we can estimate the upper bound of\|Jp​\(θ∗​\(λθ=c\)\)−Jp​\(θ∗​\(λθ=d\)\)\|\\left\|J\_\{p\}\\left\(\\theta^\{\*\}\(\\lambda^\{\\theta\}=c\)\\right\)\-J\_\{p\}\\left\(\\theta^\{\*\}\(\\lambda^\{\\theta\}=d\)\\right\)\\right\|and\|Jq​\(ϕ∗​\(λϕ=c\)\)−Jq​\(ϕ∗​\(λϕ=d\)\)\|\\left\|J\_\{q\}\\left\(\\phi^\{\*\}\(\\lambda^\{\\phi\}=c\)\\right\)\-J\_\{q\}\\left\(\\phi^\{\*\}\(\\lambda^\{\\phi\}=d\)\\right\)\\right\|, as long asJp​\(θ∗\)J\_\{p\}\(\\theta^\{\*\}\)andJq​\(ϕ∗\)J\_\{q\}\(\\phi^\{\*\}\)are well\-defined forλθ\\lambda^\{\\theta\}andλϕ\\lambda^\{\\phi\}, respectively, taking values atccanddd, which means that the system will converge to the stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)withλθ∈\{c,d\}\\lambda^\{\\theta\}\\in\\\{c,d\\\}andλϕ∈\{c,d\}\\lambda^\{\\phi\}\\in\\\{c,d\\\}\. In the main paper, we chooseλθ=λℋθ,λϕ=λℋϕ\\lambda^\{\\theta\}=\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda^\{\\phi\}=\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}as a special case\.[Theorem4\.8](https://arxiv.org/html/2605.29267#S4.Thmtheorem8)and its extensions demonstrate that in multi\-model self\-consuming systems with model interactions, if the system can converge to a stable point after iterative retraining, the final evaluation rewards for each modelJp​\(θ∗\),Jq​\(ϕ∗\)J\_\{p\}\(\\theta^\{\*\}\),J\_\{q\}\(\\phi^\{\*\}\)will not change significantly due to variations in the proportion of data sources\.

Moreover, according to the theorem proof in[SectionF\.9](https://arxiv.org/html/2605.29267#A6.SS9), the upper bound is related to models and rewards architecture, the loss functions, the real datasets, and model interaction strength\. The upper bounds also decreases as the proportion of real data increases\.

## Appendix FProofs

###### Lemma F\.1\.

\(Kantorovich\-Rubinstein\) A distribution mapD​\(m\),m∈ℳD\(m\),\\ m\\in\\mathcal\{M\}isε\\varepsilon\-sensitive if and only if for anym,m′∈ℳm,m^\{\\prime\}\\in\\mathcal\{M\}:

supg:ℝn→ℝ,g∈L​i​p1​\|𝔼Z∼D​\(m\)​g​\(Z\)−𝔼Z∼D​\(m′\)​g​\(Z\)\|≤ε​∥m−m′∥\.\\underset\{g:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\},g\\in Lip^\{1\}\}\{\\sup\}\\big\|\\mathbb\{E\}\_\{Z\\sim D\(m\)\}g\(Z\)\-\\mathbb\{E\}\_\{Z\\sim D\(m^\{\\prime\}\)\}g\(Z\)\\big\|\\leq\\varepsilon\\lVert m\-m^\{\\prime\}\\rVert\.

### F\.1Proof of Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6)

Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6)\.Defineκ≜max⁡\(γθ​Lϕ​εϕ\+2​γϕ​Lθ​εθγϕ​\(γθ−Lθ​εθ\),γϕ​Lθ​εθ\+2​γθ​Lϕ​εϕγθ​\(γϕ−Lϕ​εϕ\),Lθ​εθγθ\+Lϕ​εϕγϕ\)\.\\kappa\\triangleq\\max\\left\(\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\+2\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\phi\}\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\+2\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)\},\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\right\)\.Under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2),[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3), and[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5), ifκ<1\\kappa<1, then the stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)exists, and iterative training loop \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\) will drive\(θt,ϕt\)\(\\theta\_\{t\},\\phi\_\{t\}\)to converge to\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)at a linear rate:

‖\(θt,ϕt\)−\(θ∗,ϕ∗\)‖≤κt​‖\(θ0,ϕ0\)−\(θ∗,ϕ∗\)‖\.\|\|\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\|\|\\leq\\kappa^\{t\}\|\|\(\\theta\_\{0\},\\phi\_\{0\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\|\|\.
###### Proof\.

SinceP,QP,Qareεθ\\varepsilon\_\{\\theta\},εϕ\\varepsilon\_\{\\phi\}\-sensitive \(Assump\.[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)\), by its definition and Kantorovich\-Rubinstein \(Lemma[F\.1](https://arxiv.org/html/2605.29267#A6.Thmtheorem1)\), we have

\|𝔼z∼P​\(θ,ϕ\)​g​\(z\)−𝔼z∼P​\(θ′,ϕ′\)​g​\(z\)\|≤εθ​∥\(θ,ϕ\)−\(θ′,ϕ′\)∥,\\displaystyle\\big\|\\mathbb\{E\}\_\{z\\sim P\(\\theta,\\phi\)\}g\(z\)\-\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\\prime\},\\phi^\{\\prime\}\)\}g\(z\)\\big\|\\leq\\varepsilon\_\{\\theta\}\\lVert\(\\theta,\\phi\)\-\(\\theta^\{\\prime\},\\phi^\{\\prime\}\)\\rVert,\(15\)\|𝔼z∼Q​\(θ,ϕ\)​g​\(z\)−𝔼z∼Q​\(θ′,ϕ′\)​g​\(z\)\|≤εϕ​∥\(θ,ϕ\)−\(θ′,ϕ′\)∥,\\displaystyle\\big\|\\mathbb\{E\}\_\{z\\sim Q\(\\theta,\\phi\)\}g\(z\)\-\\mathbb\{E\}\_\{z\\sim Q\(\\theta^\{\\prime\},\\phi^\{\\prime\}\)\}g\(z\)\\big\|\\leq\\varepsilon\_\{\\phi\}\\lVert\(\\theta,\\phi\)\-\(\\theta^\{\\prime\},\\phi^\{\\prime\}\)\\rVert,for∀θ,ϕ,θ′,ϕ′\\forall\\ \\theta,\\phi,\\theta^\{\\prime\},\\phi^\{\\prime\}andg∈L​i​p1:ℝn→ℝg\\in Lip^\{1\}:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}the unions of all 1\-Lipchitz functions\. Letz=\(x,y\)z=\(x,y\)be the data pair and notice that the loss gradients∇θℓθ​\(z\),∇ϕℓϕ​\(z\)\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\),\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(z\)areLθL\_\{\\theta\},LϕL\_\{\\phi\}\-Lipschitz inzzfor anyθ\\theta,ϕ\\phi, respectively \(Assump\.[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)\), so for any vectorvvfixed,

\|∇θℓθ​\(z1\)​vTLθ−∇θℓθ​\(z2\)​vTLθ\|≤∥v∥​∥∇θℓθ​\(z1\)−∇θℓθ​\(z2\)∥\|Lθ\|≤∥v∥​∥z1−z2∥\.\|\\frac\{\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{1\}\)v^\{T\}\}\{L\_\{\\theta\}\}\-\\frac\{\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{2\}\)v^\{T\}\}\{L\_\{\\theta\}\}\|\\leq\\lVert v\\rVert\\frac\{\\lVert\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{1\}\)\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{2\}\)\\rVert\}\{\|L\_\{\\theta\}\|\}\\leq\\lVert v\\rVert\\lVert z\_\{1\}\-z\_\{2\}\\rVert\.Therefore,∇θℓθ​\(z\)​vTLθ​∥v∥\\frac\{\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\)v^\{T\}\}\{L\_\{\\theta\}\\lVert v\\rVert\}and∇ϕℓϕ​\(z\)​vTLϕ​∥v∥\\frac\{\\nabla\_\{\\phi\}\\ell\_\{\\phi\}\(z\)v^\{T\}\}\{L\_\{\\phi\}\\lVert v\\rVert\}are 1\-Lipschitz inzzfor∀θ,ϕ\\forall\\ \\theta,\\phiand fixedvv\. Replaceggwith these functions in Eq\. \([15](https://arxiv.org/html/2605.29267#A6.E15)\) and for simplicity, let

Rp​\(θ,P​\(θ^,ϕ^\)\)=𝔼z∼P​\(θ^,ϕ^\)​ℓθ​\(z\),Rq​\(ϕ,Q​\(θ^,ϕ^\)\)=𝔼z∼Q​\(θ^,ϕ^\)​ℓϕ​\(z\)\.R\_\{p\}\(\\theta,P\(\\hat\{\\theta\},\\hat\{\\phi\}\)\)=\\mathbb\{E\}\_\{z\\sim P\(\\hat\{\\theta\},\\hat\{\\phi\}\)\}\\ell\_\{\\theta\}\(z\),\\ R\_\{q\}\(\\phi,Q\(\\hat\{\\theta\},\\hat\{\\phi\}\)\)=\\mathbb\{E\}\_\{z\\sim Q\(\\hat\{\\theta\},\\hat\{\\phi\}\)\}\\ell\_\{\\phi\}\(z\)\.\(16\)Then, for any fixed distributionsP​\(θ1,ϕ1\),P​\(θ2,ϕ2\),Q​\(θ1,ϕ1\),Q​\(θ2,ϕ2\)P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\),Q\(\\theta\_\{1\},\\phi\_\{1\}\),Q\(\\theta\_\{2\},\\phi\_\{2\}\), fixed vectorvvand∀θ,ϕ\\forall\\ \\theta,\\phi,

∥\(∇θRp​\(θ,P​\(θ1,ϕ1\)\)−∇θRp​\(θ,P​\(θ2,ϕ2\)\)\)​vT∥≤Lθ​εθ​∥v∥​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥,\\displaystyle\\Big\\lVert\\Big\(\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\Big\)v^\{T\}\\Big\\rVert\\leq L\_\{\\theta\}\\varepsilon\_\{\\theta\}\\lVert v\\rVert\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert,\(17\)∥\(∇ϕRq​\(ϕ,Q​\(θ1,ϕ1\)\)−∇ϕRq​\(ϕ,Q​\(θ2,ϕ2\)\)\)​vT∥≤Lϕ​εϕ​∥v∥​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥,\\displaystyle\\Big\\lVert\\Big\(\\nabla\_\{\\phi\}R\_\{q\}\\big\(\\phi,Q\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\phi\}R\_\{q\}\\big\(\\phi,Q\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\Big\)v^\{T\}\\Big\\rVert\\leq L\_\{\\phi\}\\varepsilon\_\{\\phi\}\\lVert v\\rVert\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert,
First, we prove that for any fixed distributionsP​\(θ1,ϕ1\),P​\(θ2,ϕ2\)P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\),

∥argmin𝜃​Rp​\(θ,P​\(θ1,ϕ1\)\)−argmin𝜃​Rp​\(θ,P​\(θ2,ϕ2\)\)∥≤Lθ​εθγθ​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\.\\big\\lVert\\underset\{\\theta\}\{\\text\{argmin\}\}\\ R\_\{p\}\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\)\-\\underset\{\\theta\}\{\\text\{argmin\}\}\\ R\_\{p\}\(\\theta,P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\big\\rVert\\leq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\.Notice thatRp​\(θ,P​\(θ1,ϕ1\)\)R\_\{p\}\\big\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)isγθ\\gamma\_\{\\theta\}\-strongly convex inθ\\thetafor any fixedθ1,ϕ1\\theta\_\{1\},\\phi\_\{1\}, and this can be derived by taking expectations of both sides in Eq\. \([6](https://arxiv.org/html/2605.29267#S3.E6)\) \(Assump\.[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\)\. There exist minimal pointsφ1,φ2\\varphi\_\{1\},\\varphi\_\{2\}so that∇θRp​\(θ,P​\(θ1,ϕ1\)\)\|θ=φ1=∇θRp​\(θ,P​\(θ2,ϕ2\)\)\|θ=φ2=0\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\big\|\_\{\\theta=\\varphi\_\{1\}\}=\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\big\|\_\{\\theta=\\varphi\_\{2\}\}=0\. Therefore,

0=\(∇θRp​\(φ1,P​\(θ1,ϕ1\)\)−∇θRp​\(φ2,P​\(θ1,ϕ1\)\)\)\+∇θRp​\(φ2,P​\(θ1,ϕ1\)\)−∇θRp​\(φ2,P​\(θ2,ϕ2\)\)\.0=\\Big\(\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\Big\)\+\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\.\(18\)Combined with strongly convexity, we have

∇θRp\(φ1\\displaystyle\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(θ1,ϕ1\)\)−∇θRp\(φ2,P\(θ1,ϕ1\)\)≥∇θRp\(φ2,P\(θ1,ϕ1\)\)T\(φ1−φ2\)\+γθ2∥φ1−φ2∥2\\displaystyle,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\geq\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)^\{T\}\(\\varphi\_\{1\}\-\\varphi\_\{2\}\)\+\\frac\{\\gamma\_\{\\theta\}\}\{2\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\}\(19\)∇θRp​\(φ2,P​\(θ1,ϕ1\)\)−∇θRp​\(φ1,P​\(θ1,ϕ1\)\)≥γθ2​∥φ1−φ2∥2=γθ2​∥φ1−φ2∥2,\\displaystyle\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\geq\\frac\{\\gamma\_\{\\theta\}\}\{2\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\}=\\frac\{\\gamma\_\{\\theta\}\}\{2\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\},the second line of the formula is due to∇θRp​\(φ1,P​\(θ1,ϕ1\)\)T​\(φ2−φ1\)=0\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)^\{T\}\(\\varphi\_\{2\}\-\\varphi\_\{1\}\)=0\. Add the two formulas in Eq\. \([19](https://arxiv.org/html/2605.29267#A6.E19)\) and get0≥−γθ​∥φ1−φ2∥2≥\(φ1−φ2\)T​∇θRp​\(φ2,P​\(θ1,ϕ1\)\)=\(φ1−φ2\)T​\(∇θRp​\(φ2,P​\(θ1,ϕ1\)\)−∇θRp​\(φ1,P​\(θ1,ϕ1\)\)\)0\\geq\-\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\}\\geq\(\\varphi\_\{1\}\-\\varphi\_\{2\}\)^\{T\}\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)=\(\\varphi\_\{1\}\-\\varphi\_\{2\}\)^\{T\}\\big\(\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\big\)\. Therefore, we have∥\(φ1−φ2\)T​\(∇θRp​\(φ2,P​\(θ1,ϕ1\)\)−∇θRp​\(φ1,P​\(θ1,ϕ1\)\)\)∥≥γθ​∥φ1−φ2∥2\\big\\lVert\(\\varphi\_\{1\}\-\\varphi\_\{2\}\)^\{T\}\\Big\(\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\Big\)\\big\\rVert\\geq\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\}\. Combined with Eq\. \([18](https://arxiv.org/html/2605.29267#A6.E18)\), we have

∥\(φ1−φ2\)T\(∇\\displaystyle\\big\\lVert\(\\varphi\_\{1\}\-\\varphi\_\{2\}\)^\{T\}\\Big\(\\nablaRpθ\(φ2,P\(θ1,ϕ1\)\)−∇θRp\(φ2,P\(θ2,ϕ2\)\)\)∥≥γθ∥φ1−φ2∥2\\displaystyle\{\}\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\Big\)\\big\\rVert\\geq\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\}\(20\)⇒Lθ​εθ​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥≥γθ​∥φ1−φ2∥​by Eq\. \([17](https://arxiv.org/html/2605.29267#A6.E17)\)\\displaystyle\\Rightarrow\\ L\_\{\\theta\}\\varepsilon\_\{\\theta\}\\big\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\\rVert\\geq\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert\\ \\text\{by Eq\.~\(\\ref\{eq:apdix:RpRqDualcond\}\)\}⇒∥argmin𝜃\\displaystyle\\Rightarrow\\ \\big\\lVert\\underset\{\\theta\}\{\\text\{argmin\}\}Rp\(θ,P\(θ1,ϕ1\)\)−argmin𝜃Rp\(θ,P\(θ2,ϕ2\)\)∥≤Lθ​εθγθ∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥,\\displaystyle\\ R\_\{p\}\\big\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\underset\{\\theta\}\{\\text\{argmin\}\}\\ R\_\{p\}\\big\(\\theta,P\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\big\\rVert\\leq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\big\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\\rVert,by the definitions ofφ1,φ2\\varphi\_\{1\},\\varphi\_\{2\}\. Similarly, for modelqqit’s can be proven that for∀θ1,θ2,ϕ1,ϕ2\\forall\\ \\theta\_\{1\},\\theta\_\{2\},\\phi\_\{1\},\\phi\_\{2\},∥argminϕ​Rq​\(ϕ,Q​\(θ1,ϕ1\)\)−argminϕ​Rq​\(ϕ,Q​\(θ2,ϕ2\)\)∥≤Lϕ​εϕγϕ​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\\big\\lVert\\underset\{\\phi\}\{\\text\{argmin\}\}\\ R\_\{q\}\\big\(\\phi,Q\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\underset\{\\phi\}\{\\text\{argmin\}\}\\ R\_\{q\}\\big\(\\phi,Q\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\big\\rVert\\leq\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\big\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\\rVert\.

Next, consider the iterating process \(Eq\. \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\)\) based on distribution parameterθ,ϕ\\theta,\\phi\. LetGp​\(θ,ϕ\)=argminθ′​Rp​\(θ′,P​\(θ,ϕ\)\)G\_\{p\}\(\\theta,\\phi\)=\\underset\{\\theta^\{\\prime\}\}\{\\text\{argmin\}\}\\ R\_\{p\}\\big\(\\theta^\{\\prime\},P\(\\theta,\\phi\)\\big\)andGq​\(θ,ϕ\)=argminϕ′​Rq​\(ϕ′,Q​\(θ,ϕ\)\)G\_\{q\}\(\\theta,\\phi\)=\\underset\{\\phi^\{\\prime\}\}\{\\text\{argmin\}\}\\ R\_\{q\}\\big\(\\phi^\{\\prime\},Q\(\\theta,\\phi\)\\big\)\. The iteration process can be regarded as three cases:

\(θt\+1,ϕt\+1\\displaystyle\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)=G^1\(θt,ϕt\)≜\(Gp\(θt,ϕt\),Gq\(Gp\(θt,ϕt\),ϕt\)\),\\displaystyle\)=\\hat\{G\}\_\{1\}\(\\theta\_\{t\},\\phi\_\{t\}\)\\triangleq\\big\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),G\_\{q\}\\big\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\}\\big\)\\big\),\(21\)\(θt\+1,ϕt\+1\\displaystyle\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)=G^2\(θt,ϕt\)≜\(Gp\(θt,Gq\(θt,ϕt\)\),Gq\(θt,ϕt\)\),\\displaystyle\)=\\hat\{G\}\_\{2\}\(\\theta\_\{t\},\\phi\_\{t\}\)\\triangleq\\big\(G\_\{p\}\(\\theta\_\{t\},G\_\{q\}\(\\theta\_\{t\},\\phi\_\{t\}\)\),G\_\{q\}\\big\(\\theta\_\{t\},\\phi\_\{t\}\\big\)\\big\),\(22\)\(θt\+1\\displaystyle\(\\theta\_\{t\+1\},ϕt\+1\)=G^3\(θt,ϕt\)≜\(Gp\(θt,ϕt\),Gq\(θt,ϕt\)\),\\displaystyle,\\phi\_\{t\+1\}\)=\\hat\{G\}\_\{3\}\(\\theta\_\{t\},\\phi\_\{t\}\)\\triangleq\\big\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),G\_\{q\}\\big\(\\theta\_\{t\},\\phi\_\{t\}\\big\)\\big\),\(23\)There are a total of 9 cases if we consider two adjacent iteration rounds\. If botht\+1t\+1th round andttth round follow the same iteration pattern\. For case 1 \(Eq\. \([21](https://arxiv.org/html/2605.29267#A6.E21)\)\), notice that

∥G^1​\(θt\+1,ϕt\+1\)−G^1​\(θt,ϕt\)∥≤∥Gp​\(θt\+1,ϕt\+1\)−Gp​\(θt,ϕt\)∥⏟\(I\)\+∥Gq​\(Gp​\(θt\+1,ϕt\+1\),ϕt\+1\)−Gq​\(Gp​\(θt,ϕt\),ϕt\)∥⏟\(II\)\.\\big\\lVert\\hat\{G\}\_\{1\}\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\\hat\{G\}\_\{1\}\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert\\leq\\underbrace\{\\big\\lVert G\_\{p\}\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert\}\_\{\(\\text\{I\}\)\}\+\\underbrace\{\\big\\lVert G\_\{q\}\(G\_\{p\}\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\),\\phi\_\{t\+1\}\)\-G\_\{q\}\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\}\)\\big\\rVert\}\_\{\(\\text\{II\}\)\}\.For term \(I\), we can get\(I\)≤Lθ​εθγθ​∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥\\text\{\(I\)\}\\leq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVertby Eq\. \([20](https://arxiv.org/html/2605.29267#A6.E20)\)\. For term \(II\),

\(II\)≤∥Gq​\(Gp​\(θt\+1,ϕt\+1\),ϕt\+1\)−Gq​\(Gp​\(θt,ϕt\),ϕt\+1\)∥\+∥Gq​\(Gp​\(θt,ϕt\),ϕt\+1\)−Gq​\(Gp​\(θt,ϕt\),ϕt\)∥\\displaystyle\\leq\\big\\lVert G\_\{q\}\\big\(G\_\{p\}\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\),\\phi\_\{t\+1\}\\big\)\-G\_\{q\}\\big\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\+1\}\\big\)\\big\\rVert\+\\big\\lVert G\_\{q\}\\big\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\+1\}\\big\)\-G\_\{q\}\\big\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\}\\big\)\\big\\rVert≤Lϕ​εϕγϕ​∥Gp​\(θt\+1,ϕt\+1\)−Gp​\(θt,ϕt\)∥\+Lϕ​εϕγϕ​∥ϕt\+1−ϕt∥\\displaystyle\\leq\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\big\\lVert G\_\{p\}\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\big\\lVert\\phi\_\{t\+1\}\-\\phi\_\{t\}\\big\\rVert≤Lϕ​εϕγϕ​Lθ​εθγθ​∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥\+Lϕ​εϕγϕ​∥ϕt\+1−ϕt∥≤\(Lθ​εθ​Lϕ​εϕγθ​γϕ\+Lϕ​εϕγϕ\)​∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥\.\\displaystyle\\leq\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\big\\lVert\\phi\_\{t\+1\}\-\\phi\_\{t\}\\big\\rVert\\leq\\Big\(\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\Big\)\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert\.The derivation of \(II\)’s upper bound is also based on Eq\. \([20](https://arxiv.org/html/2605.29267#A6.E20)\)\. By combining the upper bounds of term \(I\) and \(II\), we can show that∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥=∥G^1​\(θt,ϕt\)−G^1​\(θt−1,ϕt−1\)∥≤\(Lθ​εθγθ\+Lθ​εθ​Lϕ​εϕγθ​γϕ\+Lϕ​εϕγϕ\)​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert=\\big\\lVert\\hat\{G\}\_\{1\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-\\hat\{G\}\_\{1\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert\\leq\\big\(\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\big\)\\big\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert\. For case 2 \(Eq\. \([22](https://arxiv.org/html/2605.29267#A6.E22)\)\), similarly we can prove that∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥=∥G^2​\(θt,ϕt\)−G^2​\(θt−1,ϕt−1\)∥≤\(Lθ​εθγθ\+Lθ​εθ​Lϕ​εϕγθ​γϕ\+Lϕ​εϕγϕ\)​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert=\\big\\lVert\\hat\{G\}\_\{2\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-\\hat\{G\}\_\{2\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert\\leq\\big\(\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\big\)\\big\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert\. For case 3 \(Eq\. \([23](https://arxiv.org/html/2605.29267#A6.E23)\)\), we can show that

∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥\\displaystyle\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert=∥G^3​\(θt,ϕt\)−G^3​\(θt−1,ϕt−1\)∥\\displaystyle=\\big\\lVert\\hat\{G\}\_\{3\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-\\hat\{G\}\_\{3\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert≤∥Gp​\(θt,ϕt\)−Gp​\(θt−1,ϕt−1\)∥\+∥Gq​\(θt,ϕt\)−Gq​\(θt−1,ϕt−1\)∥\\displaystyle\\leq\\big\\lVert G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-G\_\{p\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert\+\\big\\lVert G\_\{q\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-G\_\{q\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert≤Lθ​εθγθ​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\+Lϕ​εϕγϕ​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\\displaystyle\\leq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert<\(Lθ​εθγθ\+Lθ​εθ​Lϕ​εϕγθ​γϕ\+Lϕ​εϕγϕ\)​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\.\\displaystyle<\(\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\)\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert\.Therefore, for all the 3 cases if 2 adjacent rounds follow the same iteration pattern,∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥≤\(Lθ​εθγθ\+Lθ​εθ​Lϕ​εϕγθ​γϕ\+Lϕ​εϕγϕ\)​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\rVert\\leq\(\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\)\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert\.

Next, if adjacent rounds’ iteration case are different, for example\(θt\+1,ϕt\+1\)=G^1​\(θt,ϕt\)\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)=\\hat\{G\}\_\{1\}\(\\theta\_\{t\},\\phi\_\{t\}\)and\(θt,ϕt\)=G^2​\(θt,ϕt\)\(\\theta\_\{t\},\\phi\_\{t\}\)=\\hat\{G\}\_\{2\}\(\\theta\_\{t\},\\phi\_\{t\}\), we can get

∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥\\displaystyle\\big\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\big\\rVert=∥\(Gp​\(θt,ϕt\),Gq​\(Gp​\(θt,ϕt\),ϕt\)\)−\(Gp​\(θt−1,Gq​\(θt−1,ϕt−1\)\),Gq​\(θt−1,ϕt−1\)\)∥\\displaystyle=\\big\\lVert\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),G\_\{q\}\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\}\)\)\-\(G\_\{p\}\(\\theta\_\{t\-1\},G\_\{q\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\),G\_\{q\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\)\\big\\rVert\(24\)≤∥Gp\(θt,ϕt\)−Gp\(θt−1,Gq\(θt−1,ϕt−1\)\)∥\+∥Gq\(Gp\(θt,ϕt\),ϕt\)\)−Gq\(θt−1,ϕt−1\)∥\\displaystyle\\leq\\big\\lVert G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-G\_\{p\}\(\\theta\_\{t\-1\},G\_\{q\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\)\\big\\rVert\+\\big\\lVert G\_\{q\}\(G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\),\\phi\_\{t\}\)\)\-G\_\{q\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\big\\rVert≤Lθ​εθγθ​\(∥θt−θt−1∥\+∥ϕt−Gq​\(θt−1,ϕt−1\)∥⏟=0\)\+Lϕ​εϕγϕ​\(∥Gp​\(θt,ϕt\)−θt−1⏟=θt\+1−θt−1∥\+∥ϕt−ϕt−1∥\)\\displaystyle\\leq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\(\\lVert\\theta\_\{t\}\-\\theta\_\{t\-1\}\\rVert\+\\underbrace\{\\lVert\\phi\_\{t\}\-G\_\{q\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert\}\_\{=0\}\)\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\(\\lVert\\underbrace\{G\_\{p\}\(\\theta\_\{t\},\\phi\_\{t\}\)\-\\theta\_\{t\-1\}\}\_\{=\\theta\_\{t\+1\}\-\\theta\_\{t\-1\}\}\\rVert\+\\lVert\\phi\_\{t\}\-\\phi\_\{t\-1\}\\rVert\)≤\(Lθ​εθγθ\+2​Lϕ​εϕγϕ\)​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\+Lϕ​εϕγϕ​∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥\.\\displaystyle\\leq\(\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{2L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\)\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\rVert\.IfLϕ​εϕγϕ<1\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}<1, then

∥\(θt\+1,ϕt\+1\)−\(θt,ϕt\)∥≤γϕ​Lθ​εθ\+2​γθ​Lϕ​εϕγθ​\(γϕ−Lϕ​εϕ\)​∥\(θt,ϕt\)−\(θt−1,ϕt−1\)∥\.\\lVert\(\\theta\_\{t\+1\},\\phi\_\{t\+1\}\)\-\(\\theta\_\{t\},\\phi\_\{t\}\)\\rVert\\leq\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\+2\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)\}\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\\rVert\.Similarly, we can prove the compression parameter for all 9 cases shown in Table[1](https://arxiv.org/html/2605.29267#A6.T1)\. It’s easy to check that ifLϕ​εϕγϕ<1\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}<1andLθ​εθγθ<1\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}<1, thenmax​\(γθ​Lϕ​εϕ\+2​γϕ​Lθ​εθγϕ​\(γθ−Lθ​εθ\),γϕ​Lθ​εθ\+2​γθ​Lϕ​εϕγθ​\(γϕ−Lϕ​εϕ\)\)≥Lθ​εθγθ\+Lθ​εθ​Lϕ​εϕγθ​γϕ\+Lϕ​εϕγϕ\>0\\text\{max\}\(\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\+2\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\phi\}\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\+2\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)\}\)\\geq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\>0\.

Table 1:Compression parameter for 9 possible iteration cases\.Ifκ=max​\(γθ​Lϕ​εϕ\+2​γϕ​Lθ​εθγϕ​\(γθ−Lθ​εθ\),γϕ​Lθ​εθ\+2​γθ​Lϕ​εϕγθ​\(γϕ−Lϕ​εϕ\),Lθ​εθγθ\+Lϕ​εϕγϕ\)<1\\kappa=\\text\{max\}\(\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\+2\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\phi\}\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\+2\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)\},\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\)<1, then by Banach’s fixed point theorem, the stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)under our iteration setting exists and\(θ∗,ϕ∗\)=G^1​\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)=\\hat\{G\}\_\{1\}\(\\theta^\{\*\},\\phi^\{\*\}\)\. This also proves that the stable point satisfies Eq\. \([3\.1](https://arxiv.org/html/2605.29267#S3.Ex3)\)\. To prove the convergence rate, notice that

∥\(θt,ϕt\)−\(θ∗,ϕ∗\)∥\\displaystyle\\big\\lVert\(\\theta\_\{t\},\\phi\_\{t\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\\big\\rVert=∥G^1​\(θt−1,ϕt−1\)−G^1​\(θ∗,ϕ∗\)∥\\displaystyle=\\big\\lVert\\hat\{G\}\_\{1\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\-\\hat\{G\}\_\{1\}\(\\theta^\{\*\},\\phi^\{\*\}\)\\big\\rVert≤κ​∥\(θt−1,ϕt−1\)−\(θ∗,ϕ∗\)∥≤…≤κt​∥\(θ0,ϕ0\)−\(θ∗,ϕ∗\)∥\.\\displaystyle\\leq\\kappa\\big\\lVert\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\\big\\rVert\\leq\\ \.\.\.\\ \\leq\\kappa^\{t\}\\big\\lVert\(\\theta\_\{0\},\\phi\_\{0\}\)\-\(\\theta^\{\*\},\\phi^\{\*\}\)\\big\\rVert\.∎

### F\.2Proof of Proposition[4\.4](https://arxiv.org/html/2605.29267#S4.Thmtheorem4)

Proposition[4\.4](https://arxiv.org/html/2605.29267#S4.Thmtheorem4)\.Under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6),∇θ𝐅p\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\},∇ϕ𝐅q\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\},SpS\_\{p\}andSqS\_\{q\}are all invertible, and∇θ𝐅p⪰\(γθ−Lθ​εθ\)​I,∇ϕ𝐅q⪰\(γϕ−Lϕ​εϕ\)​I\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)I,\\ \\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\\succeq\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)IandSp−1=∇θ𝐅p−∇ϕ𝐅p​∇ϕ𝐅q−1​∇θ𝐅q⪰\(γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\)​I≻0,Sq−1=∇ϕ𝐅q−∇θ𝐅q​∇θ𝐅p−1​∇ϕ𝐅p⪰\(γϕ−Lϕ​εϕ−Lθ​εθ​Lϕ​εϕγθ−Lθ​εθ\)​I≻0S\_\{p\}^\{\-1\}=\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\-\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}^\{\-1\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\)I\\succ 0,\\ S\_\{q\}^\{\-1\}=\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\-\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}^\{\-1\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\succeq\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\)I\\succ 0andIIis the identity matrix\.

###### Proof\.

From Thm\. \([3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6)\), sinceκ=max​\(γθ​Lϕ​εϕ\+2​γϕ​Lθ​εθγϕ​\(γθ−Lθ​εθ\),γϕ​Lθ​εθ\+2​γθ​Lϕ​εϕγθ​\(γϕ−Lϕ​εϕ\),Lθ​εθγθ\+Lϕ​εϕγϕ\)<1\\kappa=\\text\{max\}\(\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\+2\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\phi\}\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\varepsilon\_\{\\theta\}\+2\\gamma\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)\},\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\{\+\}\\frac\{L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\)<1, it’s easy to check thatγθ−Lθ​εθ\>0\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\>0,γϕ−Lϕ​εϕ\>0\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\>0andγθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\>0\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\>0,γϕ−Lϕ​εϕ−Lθ​εθ​Lϕ​εϕγθ−Lθ​εθ\>0\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\>0\.

First, we prove that∇θ𝐅p⪰\(γθ−Lθ​εθ\)​I,∇ϕ𝐅q⪰\(γϕ−Lϕ​εϕ\)​I\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)I,\\ \\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\\succeq\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)I\. For any vectorv≠0v\\neq 0, by∇θ𝐅p\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}’s definition we decompose the matrix into different dimensions and can get

v​∇θ𝐅p​vT\\displaystyle v\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}v^\{T\}=v​∂∂θ​𝔼z∼P​\(θ,ϕ\)​∇θℓθ​\(z;θ\)\|\(θ,ϕ\)=\(θ∗,ϕ∗\)​vT\\displaystyle=v\\frac\{\\partial\}\{\\partial\\theta\}\\mathbb\{E\}\_\{z\\sim P\(\\theta,\\phi\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta\)\\big\|\_\{\(\\theta,\\phi\)=\(\\theta^\{\*\},\\phi^\{\*\}\)\}v^\{T\}\(25\)=𝔼z∼P​\(θ∗,ϕ∗\)​\[v​∇θ2ℓθ​\(z;θ∗\)​vT\]\+∂∂t​𝔼z∼P​\(θ∗\+t​v,ϕ∗\)​\[∇θℓθ​\(z;θ∗\)​vT\]\|t=0\.\\displaystyle=\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\big\[v\\nabla\_\{\\theta\}^\{2\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\+\\frac\{\\partial\}\{\\partial t\}\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\}\+tv,\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\\big\|\_\{t=0\}\.For the first term in Eq\. \([25](https://arxiv.org/html/2605.29267#A6.E25)\), sinceℓθ\\ell\_\{\\theta\}isγθ\\gamma\_\{\\theta\}\-strong convex \(Assump\. \([3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\)\), it’s easy to check that𝔼z∼P​\(θ∗,ϕ∗\)​∇θ2ℓθ​\(z;θ∗\)⪰γθ​I\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\nabla\_\{\\theta\}^\{2\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\succeq\\gamma\_\{\\theta\}IwhereIIis the identity matrix\. Therefore,𝔼z∼P​\(θ∗,ϕ∗\)​\[v​∇θ2ℓθ​\(z;θ∗\)​vT\]≥γθ​∥v∥2\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\big\[v\\nabla\_\{\\theta\}^\{2\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\\geq\\gamma\_\{\\theta\}\\lVert v\\rVert^\{2\}\. For the second term in Eq\. \([25](https://arxiv.org/html/2605.29267#A6.E25)\),

\|∂∂t𝔼z∼P​\(θ∗\+t​v,ϕ∗\)\[∇θℓθ\(z;θ∗\)vT\]\|t=0\|\\displaystyle\\Big\|\\frac\{\\partial\}\{\\partial t\}\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\}\+tv,\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\\big\|\_\{t=0\}\\Big\|=\|limδ→0​𝔼z∼P​\(θ∗\+δ​v,ϕ∗\)​\[∇θℓθ​\(z;θ∗\)​vT\]−𝔼z∼P​\(θ∗,ϕ∗\)​\[∇θℓθ​\(z;θ∗\)​vT\]δ\|\\displaystyle=\\Big\|\\underset\{\\delta\\rightarrow 0\}\{\\text\{lim\}\}\\ \\frac\{\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\}\+\\delta v,\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\-\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\}\{\\delta\}\\Big\|≤sup\|δ\|≤1​\|𝔼z∼P​\(θ∗\+δ​v,ϕ∗\)​\[∇θℓθ​\(z;θ∗\)​vT\]−𝔼z∼P​\(θ∗,ϕ∗\)​\[∇θℓθ​\(z;θ∗\)​vT\]\|\|δ\|,\\displaystyle\\leq\\underset\{\|\\delta\|\\leq 1\}\{\\text\{sup\}\}\\frac\{\|\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\}\+\\delta v,\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\-\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\|\}\{\|\\delta\|\},and notice that by Assump\. \([3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)\) for anyz1,z2z\_\{1\},z\_\{2\},

\|∇θℓθ​\(z1;θ∗\)​vT−∇θℓθ​\(z2;θ∗\)​vT\|≤∥∇θℓθ​\(z1;θ∗\)−∇θℓθ​\(z2;θ∗\)∥​∥v∥≤Lθ​∥v∥​∥z1−z2∥,\|\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{1\};\\theta^\{\*\}\)v^\{T\}\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{2\};\\theta^\{\*\}\)v^\{T\}\|\\leq\\lVert\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{1\};\\theta^\{\*\}\)\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z\_\{2\};\\theta^\{\*\}\)\\rVert\\lVert v\\rVert\\leq L\_\{\\theta\}\\lVert v\\rVert\\lVert z\_\{1\}\-z\_\{2\}\\rVert,which means∇θℓθ​\(z;θ∗\)​vT:ℝn→ℝ\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}:\\mathbb\{R\}^\{n\}\\rightarrow\\mathbb\{R\}is aLθ​∥v∥L\_\{\\theta\}\\lVert v\\rVert\-Lipschitz function\. Therefore, combined with Kantorovich\-Rubinstein \(Lemma[F\.1](https://arxiv.org/html/2605.29267#A6.Thmtheorem1)\) andεθ\\varepsilon\_\{\\theta\}\-sensitivity, we can get

\|∂∂t𝔼z∼P​\(θ∗\+t​v,ϕ∗\)\[∇θℓθ\(z;θ∗\)vT\]\|t=0\|≤Lθ​∥v∥​εθ​∥δ​v∥\|δ\|=Lθεθ∥v∥2\.\\displaystyle\\Big\|\\frac\{\\partial\}\{\\partial t\}\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\}\+tv,\\phi^\{\*\}\)\}\\big\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)v^\{T\}\\big\]\\big\|\_\{t=0\}\\Big\|\\leq\\frac\{L\_\{\\theta\}\\lVert v\\rVert\\varepsilon\_\{\\theta\}\\lVert\\delta v\\rVert\}\{\|\\delta\|\}=L\_\{\\theta\}\\varepsilon\_\{\\theta\}\\lVert v\\rVert^\{2\}\.\(26\)Back to Eq\. \([25](https://arxiv.org/html/2605.29267#A6.E25)\), we have

v​∇θ𝐅p​vT≥\(γθ−Lθ​εθ\)​∥v∥2\>0\.v\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}v^\{T\}\\geq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\\lVert v\\rVert^\{2\}\>0\.Similarly, we can prove that∇ϕ𝐅q⪰\(γϕ−Lϕ​εϕ\)​I\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\\succeq\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)I\.

Next, we prove thatS^p=∇θ𝐅p−∇ϕ𝐅p​∇ϕ𝐅q−1​∇θ𝐅q⪰\(γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\)​I\\hat\{S\}\_\{p\}=\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\-\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}^\{\-1\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\)I\. The corresponding conclusion forSqS\_\{q\}can be proven symmetrically\. Consider the upper bound of∥∇θ𝐅q∥\\lVert\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\rVertand∥∇ϕ𝐅p∥\\lVert\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\rVert\. Adding small perturbationst​hthtoϕ\\phiinFpF\_\{p\},t∈ℝt\\in\\mathbb\{R\}and similar to the proof of∇θ𝐅p\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}, calculate

∥∇θ𝐅q∥\\displaystyle\\lVert\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\rVert=sup∥h∥≤1​∥h​∇θ𝐅q∥∥h∥=limt→0​sup∥h∥≤1​∥Fp​\(θ,ϕ\+t​h\)−Fp​\(θ,ϕ\)∥∥t​h∥\\displaystyle=\\underset\{\\lVert h\\rVert\\leq 1\}\{\\text\{sup\}\}\\frac\{\\lVert h\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\rVert\}\{\\lVert h\\rVert\}=\\underset\{t\\rightarrow 0\}\{\\text\{lim\}\}\\ \\underset\{\\lVert h\\rVert\\leq 1\}\{\\text\{sup\}\}\\frac\{\\big\\lVert F\_\{p\}\(\\theta,\\phi\+th\)\-F\_\{p\}\(\\theta,\\phi\)\\big\\rVert\}\{\\lVert th\\rVert\}=limt→0​sup∥h∥≤1​∥𝔼z∼P​\(θ,ϕ\+t​h\)​∇θℓθ​\(z;θ\)−𝔼z∼P​\(θ,ϕ\)​∇θℓθ​\(z;θ\)∥∥t​h∥\\displaystyle=\\underset\{t\\rightarrow 0\}\{\\text\{lim\}\}\\ \\underset\{\\lVert h\\rVert\\leq 1\}\{\\text\{sup\}\}\\frac\{\\big\\lVert\\mathbb\{E\}\_\{z\\sim P\(\\theta,\\phi\+th\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta\)\-\\mathbb\{E\}\_\{z\\sim P\(\\theta,\\phi\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta\)\\big\\rVert\}\{\\lVert th\\rVert\}≤sup∥h∥≤1​Lθ​∥𝔼z∼P​\(θ,ϕ\+h\)​∇θℓθ​\(z;θ\)Lθ−𝔼z∼P​\(θ,ϕ\)​∇θℓθ​\(z;θ\)Lθ∥∥h∥≤Lθ​εθ​∥\(θ,ϕ\+h\)−\(θ,ϕ\)∥∥h∥=Lθ​εθ\.\\displaystyle\\leq\\underset\{\\lVert h\\rVert\\leq 1\}\{\\text\{sup\}\}\\frac\{L\_\{\\theta\}\\big\\lVert\\mathbb\{E\}\_\{z\\sim P\(\\theta,\\phi\+h\)\}\\frac\{\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta\)\}\{L\_\{\\theta\}\}\-\\mathbb\{E\}\_\{z\\sim P\(\\theta,\\phi\)\}\\frac\{\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta\)\}\{L\_\{\\theta\}\}\\big\\rVert\}\{\\lVert h\\rVert\}\\leq\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}\\big\\lVert\(\\theta,\\phi\+h\)\-\(\\theta,\\phi\)\\big\\rVert\}\{\\lVert h\\rVert\}=L\_\{\\theta\}\\varepsilon\_\{\\theta\}\.\(27\)The inequality in the derivation of Eq\.[F\.2](https://arxiv.org/html/2605.29267#A6.Ex67)is due to the distributionPP’sεθ\\varepsilon\_\{\\theta\}\-sensitivity and Kantorovich\-Rubinstein \(Lemma[F\.1](https://arxiv.org/html/2605.29267#A6.Thmtheorem1)\)\. Similarly, we have∥∇θ𝐅q∥≤Lϕ​εϕ\\lVert\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\\rVert\\leq L\_\{\\phi\}\\varepsilon\_\{\\phi\}\.

Recall thatS^p=∇θ𝐅p−∇ϕ𝐅p​∇ϕ𝐅q−1​∇θ𝐅q\\hat\{S\}\_\{p\}=\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\-\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}^\{\-1\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}\. Therefore, for any vectorv≠0v\\neq 0,

vT​S^p​v\\displaystyle v^\{T\}\\hat\{S\}\_\{p\}v=vT​∇θ𝐅p​v−vT​∇ϕ𝐅p​∇ϕ𝐅q−1​∇θ𝐅q​v\\displaystyle=v^\{T\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}v\-v^\{T\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}^\{\-1\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}v≥\(γθ−Lθ​εθ\)​∥v∥2−∥vT​∇ϕ𝐅p∥​∥∇ϕ𝐅q−1∥​∥∇θ𝐅q​v∥≥\(γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\)​∥v∥2,\\displaystyle\\geq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)\\lVert v\\rVert^\{2\}\-\\lVert v^\{T\}\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\rVert\\lVert\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}^\{\-1\}\\rVert\\lVert\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{q\}v\\rVert\\geq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\)\\lVert v\\rVert^\{2\},\(28\)sinceLθ​εθ≥∥∇ϕ𝐅p∥≥∥∇ϕ𝐅p​v∥∥v∥L\_\{\\theta\}\\varepsilon\_\{\\theta\}\\geq\\lVert\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}\\rVert\\geq\\frac\{\\lVert\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{p\}v\\rVert\}\{\\lVert v\\rVert\}by the matrix norm’s definition and∇ϕ𝐅q⪰\(γϕ−Lϕ​εϕ\)​I,∇θ𝐅p⪰\(γθ−Lϕ​εϕ\)​I\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}\\succeq\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)I,\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)I\. The proof of∥∇ϕ𝐅q−1∥≤\(γϕ−Lϕ​εϕ\)−1\\lVert\\nabla\_\{\\phi\}\\mathbf\{F\}\_\{q\}^\{\-1\}\\rVert\\leq\(\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\)^\{\-1\}is similar to the proof in Corollary[4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7)\. Therefore, by Eq\. \([F\.2](https://arxiv.org/html/2605.29267#A6.Ex69)\),S^p⪰\(γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\)​I\\hat\{S\}\_\{p\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\)Iand it is invertible\. Notice thatS^p\\hat\{S\}\_\{p\}is not necessarily a symmetric matrix, and its positive definiteness differs from the traditional symmetric positive definiteness\. Similarly, we can prove thatSqS\_\{q\}is invertible\. ∎

### F\.3Proof of Proposition[4\.1](https://arxiv.org/html/2605.29267#S4.Thmtheorem1)

\[Characterization of∂θ∗∂λℋθ\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}and∂ϕ∗∂λℋθ\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\]∂θ∗∂λℋθ=−Sp​\(∂Fp∂λℋθ\+Cp​∂Fq∂λℋθ\),∂ϕ∗∂λℋθ=−Sq​\(Cq​∂Fp∂λℋθ\+∂Fq∂λℋθ\)\.\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-S\_\{p\}\\left\(\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+C\_\{p\}\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\),\\ \\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-S\_\{q\}\\left\(C\_\{q\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\)\.whereSp,SqS\_\{p\},S\_\{q\}are the sensitivity matrices andCp,CqC\_\{p\},C\_\{q\}are the cross\-model influence matrices\.

###### Proof\.

In the main paper, we already have that

∀λℋθ,Fp​\(θ∗,ϕ∗,λℋθ\)=Fq​\(θ∗,ϕ∗,λℋθ\)=0\.\\forall\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},~~F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)=F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)=0\.Differentiating the above equation with respect toλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}, we obtain:

∂Fp​\(θ∗,ϕ∗,λℋθ\)∂θ​∂θ∗​\(λℋθ\)∂λℋθ\\displaystyle\\frac\{\\partial F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\theta\}\\frac\{\\partial\\theta^\{\*\}\(\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+∂Fp​\(θ∗,ϕ∗,λℋθ\)∂ϕ​∂ϕ∗​\(λℋθ\)∂λℋθ\+∂Fp​\(θ∗,ϕ∗,λℋθ\)∂λℋθ=0,\\displaystyle\+\\frac\{\\partial F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\phi\}\\frac\{\\partial\\phi^\{\*\}\(\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=0,\(29\)∂Fq​\(θ∗,ϕ∗,λℋθ\)∂ϕ​∂ϕ∗​\(λℋθ\)∂λℋθ\\displaystyle\\frac\{\\partial F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\phi\}\\frac\{\\partial\\phi^\{\*\}\(\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+∂Fq​\(θ∗,ϕ∗,λℋθ\)∂θ​∂θ∗​\(λℋθ\)∂λℋθ\+∂Fq​\(θ∗,ϕ∗,λℋθ\)∂λℋθ=0\.\\displaystyle\+\\frac\{\\partial F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\theta\}\\frac\{\\partial\\theta^\{\*\}\(\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=0\.Combined with∇θ𝑭p\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{p\},∇θ𝑭q\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{q\}in Definition[4\.2](https://arxiv.org/html/2605.29267#S4.Thmtheorem2)and Proposition[4\.4](https://arxiv.org/html/2605.29267#S4.Thmtheorem4), solving Eq\. \([29](https://arxiv.org/html/2605.29267#A6.E29)\) yields that

∂θ∗∂λℋθ=−\(∇θ𝑭p−∇ϕ𝑭p​\(∇ϕ𝑭q\)−1​∇θ𝑭q\)−1​\(∂Fp​\(θ∗,ϕ∗,λℋθ\)∂λℋθ−∇ϕ𝑭p​\(∇ϕ𝑭q\)−1​∂Fq​\(θ∗,ϕ∗,λℋθ\)∂λℋθ\),\\displaystyle\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-\(\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{p\}\-\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{p\}\(\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{q\}\)^\{\-1\}\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{q\}\)^\{\-1\}\(\\frac\{\\partial F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\-\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{p\}\(\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{q\}\)^\{\-1\}\\frac\{\\partial F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\),∂ϕ∗∂λℋθ=−\(∇ϕ𝑭q−∇θ𝑭q​\(∇θ𝑭p\)−1​∇ϕ𝑭p\)−1​\(∂Fq​\(θ∗,ϕ∗,λℋθ\)∂λℋθ−∇θ𝑭q​\(∇θ𝑭p\)−1​∂Fp​\(θ∗,ϕ∗,λℋθ\)∂λℋθ\)\.\\displaystyle\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-\(\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{q\}\-\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{q\}\(\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{p\}\)^\{\-1\}\\nabla\_\{\\phi\}\\boldsymbol\{F\}\_\{p\}\)^\{\-1\}\(\\frac\{\\partial F\_\{q\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\-\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{q\}\(\\nabla\_\{\\theta\}\\boldsymbol\{F\}\_\{p\}\)^\{\-1\}\\frac\{\\partial F\_\{p\}\(\\theta^\{\*\},\\phi^\{\*\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\)\.∎

### F\.4Proof of Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)

Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)\.Under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5)and condition in Theorem[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6), the self\-consuming multi\-model system converges\. LetPℋP\_\{\\mathcal\{H\}\}be the distribution of human\-curated dataℋtθ\\mathcal\{H\}^\{\\theta\}\_\{t\}after convergence\.

∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=11−λℋθ​⟨∇θJp​\(θ∗\),Sp​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩,\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}\\,\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle,∂Jq​\(ϕ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=11−λℋθ​⟨∇ϕJq​\(ϕ∗\),Sq​Cq​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\\,S\_\{q\}C\_\{q\}\\,\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle\.whereSp,SqS\_\{p\},S\_\{q\}are the sensitivity matrices,CqC\_\{q\}is the cross\-model influence matrix, and⟨⋅,⋅⟩\\langle\\cdot,\\cdot\\rangledenotes the inner product between two vectors\.

###### Proof\.

For modelθ\\theta, letfθ​\(θ,ϕ\)f\_\{\\theta\}\(\\theta,\\phi\)beP​\(θ,ϕ\)P\(\\theta,\\phi\)’s density functions, andfθ,c​\(θ,ϕ\),fθ,s​\(θ,ϕ\)f\_\{\\theta,c\}\(\\theta,\\phi\),f\_\{\\theta,s\}\(\\theta,\\phi\)be the density functions ofℋθ​\(θ,ϕ\)\\mathcal\{H\}^\{\\theta\}\(\\theta,\\phi\)’s distribution,𝒮θ​\(θ,ϕ\)\\mathcal\{S\}^\{\\theta\}\(\\theta,\\phi\)’s distribution respectively\. For modelϕ\\phi, letfϕ​\(θ,ϕ\)f\_\{\\phi\}\(\\theta,\\phi\)beQ​\(θ,ϕ\)Q\(\\theta,\\phi\)’s density functions, andfϕ,c​\(θ,ϕ\),fϕ,s​\(θ,ϕ\)f\_\{\\phi,c\}\(\\theta,\\phi\),f\_\{\\phi,s\}\(\\theta,\\phi\)be the density functions ofℋϕ​\(θ,ϕ\)\\mathcal\{H\}^\{\\phi\}\(\\theta,\\phi\)’s distribution,𝒮ϕ​\(θ,ϕ\)\\mathcal\{S\}^\{\\phi\}\(\\theta,\\phi\)’s distribution respectively\. For the fixed real data distribution, letfθ,r,fϕ,rf\_\{\\theta,r\},f\_\{\\phi,r\}be the corresponding density functions\. Therefore, by the definition of the mixture distributionsPPandQQ, we have

fθ​\(θ,ϕ\)=λℛθ​fθ,r\+λ𝒮θ​fθ,s​\(θ,ϕ\)\+λℋθ​fθ,c​\(θ,ϕ\),fϕ​\(θ,ϕ\)=λℛϕ​fϕ,r\+λ𝒮ϕ​fϕ,s​\(θ,ϕ\)\+λℋϕ​fϕ,c​\(θ,ϕ\)\.f\_\{\\theta\}\(\\theta,\\phi\)=\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}f\_\{\\theta,r\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}f\_\{\\theta,s\}\(\\theta,\\phi\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}f\_\{\\theta,c\}\(\\theta,\\phi\),\\ f\_\{\\phi\}\(\\theta,\\phi\)=\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}f\_\{\\phi,r\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\}f\_\{\\phi,s\}\(\\theta,\\phi\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}f\_\{\\phi,c\}\(\\theta,\\phi\)\.\(30\)By the definition ofFpF\_\{p\}, its partial gradient ofλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}in Proposition[4\.1](https://arxiv.org/html/2605.29267#S4.Thmtheorem1)is

∂Fp∂λℋθ\\displaystyle\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=∂∂λℋθ​𝔼z=\(x,y\)∼P​\(θ∗,ϕ∗\)​∇θℓθ​\(z;θ∗\)=∂∂λℋθ​∫∇θℓθ​\(z;θ∗\)​fθ​\(z;θ∗,ϕ∗\)​𝑑z\\displaystyle=\\frac\{\\partial\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\mathbb\{E\}\_\{z=\(x,y\)\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)=\\frac\{\\partial\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)f\_\{\\theta\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)dz=∂∂λℋθ​∫∇θℓθ​\(z;θ∗\)​\(λ𝒮θ​fθ,s​\(z;θ∗,ϕ∗\)\+λℋθ​fθ,c​\(z;θ∗,ϕ∗\)\+λℛθ​fθ,r​\(z\)\)​𝑑z\.\\displaystyle=\\frac\{\\partial\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\big\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}f\_\{\\theta,s\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}f\_\{\\theta,r\}\(z\)\\big\)dz\.\(31\)We considerfθ​\(z;θ,ϕ\)=fθ​\(λ𝒮θ,λℋθ,λℛθ;z,θ,ϕ\)f\_\{\\theta\}\(z;\\theta,\\phi\)=f\_\{\\theta\}\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\};z,\\theta,\\phi\)as a function of\(λ𝒮θ,λℋθ,λℛθ\)\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\)satisfyingλℛθ\+λ𝒮θ\+λℋθ=1\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=1, and its total derivative is

d​fθ​\(λ𝒮θ,λℋθ,λℛθ;z,θ,ϕ\)=fθ,s​\(z;θ,ϕ\)​d​λ𝒮θ\+fθ,c​\(z;θ,ϕ\)​d​λℋθ\+fθ,r​\(z\)​d​λℛθ,d​λ𝒮θ\+d​λℋθ\+d​λℛθ=0\.df\_\{\\theta\}\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\};z,\\theta,\\phi\)=f\_\{\\theta,s\}\(z;\\theta,\\phi\)d\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+f\_\{\\theta,c\}\(z;\\theta,\\phi\)d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+f\_\{\\theta,r\}\(z\)d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\ d\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=0\.Since changing modelθ\\theta’s human curation strength is done by increasing or decreasing the size ofℋθ\\mathcal\{H\}^\{\\theta\}, it’s easy to check thatd​λ𝒮θ=−λ𝒮θλ𝒮θ\+λℛθ​d​λℋθd\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}=\-\\frac\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}andd​λℛθ=−λℛθλ𝒮θ\+λℛθ​d​λℋθd\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=\-\\frac\{\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\.

Therefore, by Eq\. \([F\.4](https://arxiv.org/html/2605.29267#A6.Ex75)\),

∂Fp∂λℋθ\\displaystyle\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=∫∇θℓθ​\(z;θ∗\)​d​fθ​\(λ𝒮θ,λℋθ,λℛθ;z,θ∗,ϕ∗\)d​λℋθ​𝑑z\\displaystyle=\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\frac\{df\_\{\\theta\}\(\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\};z,\\theta^\{\*\},\\phi^\{\*\}\)\}\{d\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}dz=∫∇θℓθ​\(z;θ∗\)​\(−λ𝒮θλ𝒮θ\+λℛθ​fθ,s​\(z;θ∗,ϕ∗\)\+fθ,c​\(z;θ∗,ϕ∗\)−λℛθλ𝒮θ\+λℛθ​fθ,r​\(z\)\)​𝑑z\\displaystyle=\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\big\(\-\\frac\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}f\_\{\\theta,s\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\-\\frac\{\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}f\_\{\\theta,r\}\(z\)\\big\)dz=∫∇θℓθ​\(z;θ∗\)​\(fθ,c​\(z;θ∗,ϕ∗\)λ𝒮θ\+λℛθ−λ𝒮θ​fθ,s​\(z;θ∗,ϕ∗\)\+λℋθ​fθ,c​\(z;θ∗,ϕ∗\)\+λℛθ​fθ,r​\(z\)λ𝒮θ\+λℛθ\)​𝑑z\\displaystyle=\\int\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\big\(\\frac\{f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\-\\frac\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}f\_\{\\theta,s\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}f\_\{\\theta,c\}\(z;\\theta^\{\*\},\\phi^\{\*\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}f\_\{\\theta,r\}\(z\)\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\\big\)dz=11−λℋθ​\(𝔼z∼Pℋ​∇θℓθ​\(z;θ∗\)−𝔼z∼P​\(θ∗,ϕ∗\)​∇θℓθ​\(z;θ∗\)\)=11−λℋθ​𝔼z∼Pℋ​∇θℓθ​\(θ∗\)\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\big\(\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\-\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\theta^\{\*\}\)\\big\)=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\mathbb\{E\}\_\{z\\sim P\_\{\\mathcal\{H\}\}\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\(32\)since𝔼z∼P​\(θ∗,ϕ∗\)​∇θℓθ​\(θ∗\)=0\\mathbb\{E\}\_\{z\\sim P\(\\theta^\{\*\},\\phi^\{\*\}\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)=0by the definition ofθ∗,ϕ∗\\theta^\{\*\},\\phi^\{\*\}\. Similarly, since we only change the human curation ratioλℋθ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}for modelpp, it’s easy to check that∂Fq∂λℋθ=0\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=0\.

Therefore, by the chain rule and Proposition[4\.1](https://arxiv.org/html/2605.29267#S4.Thmtheorem1), we can get

∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=∂Jp​\(θ∗\)∂θ​∂θ∗∂λℋθ=⟨∇θJp​\(θ∗\),−Sp​\(∂Fp∂λℋθ\+Cp​∂Fq∂λℋθ\)⟩\\displaystyle=\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\theta\}\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\-S\_\{p\}\\big\(\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+C\_\{p\}\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\big\)\\rangle=11−λℋθ​⟨∇θJp​\(θ∗\),Sp​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle\.Similarly, by Proposition[4\.1](https://arxiv.org/html/2605.29267#S4.Thmtheorem1)and the chain rule, we have

∂Jq​\(ϕ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=∂Jq​\(ϕ∗\)∂ϕ∗​∂ϕ∗∂λℋθ=⟨∇ϕJq​\(ϕ∗\),−Sq​\(∂Fq∂λℋθ\+Cq​∂Fp∂λℋθ\)⟩\\displaystyle=\\frac\{\\partial J\_\{q\}\(\\phi^\{\*\}\)\}\{\\partial\\phi^\{\*\}\}\\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),\-S\_\{q\}\\big\(\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+C\_\{q\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\big\)\\rangle=11−λℋθ​⟨∇ϕJq​\(ϕ∗\),Sq​Cq​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩\.\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\langle\\nabla\_\{\\phi\}J\_\{q\}\(\\phi^\{\*\}\),S\_\{q\}C\_\{q\}\\ \\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle\.∎

### F\.5Proof of Corollary[E\.1](https://arxiv.org/html/2605.29267#A5.Thmtheorem1)

Corollary[E\.1](https://arxiv.org/html/2605.29267#A5.Thmtheorem1)For a single modelθ\\theta, under Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5), ifρp≈1\\rho\_\{p\}\\approx 1, then∂Jp​\(θ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}^\{\\theta\}\}\}\>0\.

###### Proof\.

Since there is only one model, by the definitions ofSpS\_\{p\}andCpC\_\{p\},Sp=\(∇θ𝐅p\)−1S\_\{p\}=\(\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\)^\{\-1\}andCp=0C\_\{p\}=0without model interactions\. Notice that,∇θ𝐅p⪰\(γθ−Lθ​εθ\)​I≻0\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\)I\\succ 0\. Therefore,∇θ𝐅p\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}is invertible and for anyx≠0x\\neq 0, lety=Sp​x≠0y=S\_\{p\}x\\neq 0, we have

xT​Sp​x=xT​SpT​∇θ𝐅p\+∇θ𝐅pT2​Sp​x=yT​∇θ𝐅p\+∇θ𝐅pT2​y=yT​∇θ𝐅p​y\>0,x^\{T\}S\_\{p\}x=x^\{T\}S\_\{p\}^\{T\}\\frac\{\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\+\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}^\{T\}\}\{2\}S\_\{p\}x=y^\{T\}\\frac\{\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\+\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}^\{T\}\}\{2\}y=y^\{T\}\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}y\>0,which meansSp=\(∇θ𝐅p\)−1S\_\{p\}=\(\\nabla\_\{\\theta\}\\mathbf\{F\}\_\{p\}\)^\{\-1\}is also positive definite\. Moreover, sinceρp≈1\\rho\_\{p\}\\approx 1, there exists a constantc\>0c\>0such that𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]≈c​∇θJp​\(θ∗\)\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\approx c\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)which means these two vectors have the same direction\. By Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5),

∂Jp​\(θ∗\)∂λℋθ=11−λℋθ​⟨∇θJp​\(θ∗\),Sp​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩=c1−λℋθ​∇θJp​\(θ∗\)T​Sp​∇θJp​\(θ∗\)\>0,\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\,\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\,S\_\{p\}\\,\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rangle=\\frac\{c\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)^\{T\}S\_\{p\}\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)\>0,sinceSpS\_\{p\}is positive definite\. ∎

### F\.6Proof of Corollary[4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7)

Corollary[4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7)\.Briefly noteτp=γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ∈\(0,1\)\\tau\_\{p\}=\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\\in\(0,1\)and letmpm\_\{p\}be the minimal eigenvalue ofSp\+SpT2\\frac\{S\_\{p\}\+S\_\{p\}^\{T\}\}\{2\}, we have∥Sp∥≤1τp\\lVert S\_\{p\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{p\}\}and

if​\|ρp\|\>11\+mp2​τp2,then​sign⁡\(ρp\)⋅∂Jp​\(θ∗\)∂λℋθ\>0\.\\text\{if\}\\ \|\\rho\_\{p\}\|\>\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\},\\ \\text\{then\}\\ \\operatorname\{sign\}\(\\rho\_\{p\}\)\\cdot\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0\.
###### Proof\.

LetS​Tp=Sp\+SpT2ST\_\{p\}=\\frac\{S\_\{p\}\+S\_\{p\}^\{T\}\}\{2\}, andmpm\_\{p\}be its minimal eigenvalues, respectively\. First, we prove that0<mp≤∥Sp∥≤1τp0<m\_\{p\}\\leq\\lVert S\_\{p\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{p\}\}andSpS\_\{p\}is an asymmetric positive definite matrix\. By[Proposition4\.4](https://arxiv.org/html/2605.29267#S4.Thmtheorem4),Sp−1⪰τp​IS\_\{p\}^\{\-1\}\\succeq\\tau\_\{p\}Imeans that for∀v≠0,v​Sp−1​vT≥τp​∥v∥2\>0\\forall\\ v\\neq 0,\\ vS\_\{p\}^\{\-1\}v^\{T\}\\geq\\tau\_\{p\}\\lVert v\\rVert^\{2\}\>0\. Notice thatv​Sp≠0vS\_\{p\}\\neq 0, otherwise 0 is an eigenvalue ofSpS\_\{p\}which contradicts it being invertible\. Therefore, for anyv≠0v\\neq 0we have

\(v​Sp\)​Sp−1​\(v​Sp\)T=\(v​Sp\)​Sp−T​\(v​Sp\)T=v​Sp​vT=v​Sp\+SpT2​vT=v​S​Tp​vT≥τp​∥v​Sp∥2\>0\(vS\_\{p\}\)S\_\{p\}^\{\-1\}\(vS\_\{p\}\)^\{T\}=\(vS\_\{p\}\)S\_\{p\}^\{\-T\}\(vS\_\{p\}\)^\{T\}=vS\_\{p\}v^\{T\}=v\\frac\{S\_\{p\}\+S\_\{p\}^\{T\}\}\{2\}v^\{T\}=vST\_\{p\}v^\{T\}\\geq\\tau\_\{p\}\\lVert vS\_\{p\}\\rVert^\{2\}\>0indicates thatS​TpST\_\{p\}is a symmetric positive definite matrix\. Sincempm\_\{p\}isS​TpST\_\{p\}’s minimal eigenvalue,mp\>0m\_\{p\}\>0\. Moreover, for any unit vectorxx,

mp≤max∥x∥=1​x​S​Tp​xT=max∥x∥=1​x​Sp​xT≤∥x∥​∥Sp​xT∥≤∥Sp∥\.m\_\{p\}\\leq\\underset\{\\lVert x\\rVert=1\}\{\\text\{max\}\}\\ xST\_\{p\}x^\{T\}=\\underset\{\\lVert x\\rVert=1\}\{\\text\{max\}\}\\ xS\_\{p\}x^\{T\}\\leq\\lVert x\\rVert\\lVert S\_\{p\}x^\{T\}\\rVert\\leq\\lVert S\_\{p\}\\rVert\.max∥x∥=1​xT​S​Tp​x\\underset\{\\lVert x\\rVert=1\}\{\\text\{max\}\}\\ x^\{T\}ST\_\{p\}xequals the maximum eigenvalue ofS​TpST\_\{p\}, somp≤max∥x∥=1​xT​S​Tp​xm\_\{p\}\\leq\\underset\{\\lVert x\\rVert=1\}\{\\text\{max\}\}\\ x^\{T\}ST\_\{p\}x\. Furthermore, for any vectorv≠0v\\neq 0,

τp​∥v∥2≤v​Sp−1​vT≤∥v∥​∥Sp−1​vT∥⇒min∥v∥=1​∥Sp−1​vT∥≥τp,\\tau\_\{p\}\\lVert v\\rVert^\{2\}\\leq vS\_\{p\}^\{\-1\}v^\{T\}\\leq\\lVert v\\rVert\\lVert S\_\{p\}^\{\-1\}v^\{T\}\\rVert\\Rightarrow\\underset\{\\lVert v\\rVert=1\}\{\\text\{min\}\}\\ \\lVert S\_\{p\}^\{\-1\}v^\{T\}\\rVert\\geq\\tau\_\{p\},and for any vectorv≠0,∃v^​s\.t\.v=v^​Sp−1v\\neq 0,\\exists\\ \\hat\{v\}\\ s\.t\.\\ v=\\hat\{v\}S\_\{p\}^\{\-1\}andv∈ℝn/\{0\}→v^∈ℝn/\{0\}v\\in\\mathbb\{R\}^\{n\}/\\\{0\\\}\\rightarrow\\hat\{v\}\\in\\mathbb\{R\}^\{n\}/\\\{0\\\}is a bijection for somennsinceSpS\_\{p\}is invertible,

∥Sp∥=maxv≠0​∥v​Sp∥∥v∥=maxv^≠0​∥v^​Sp−1​Sp∥∥v^​Sp−1∥=maxv^≠0​∥v^∥∥v^​Sp−1∥=1minv^≠0​∥v^​Sp−1∥∥v^∥=1min∥v^∥=1​∥v^​Sp−1∥≤1τp\.\\lVert S\_\{p\}\\rVert=\\underset\{v\\neq 0\}\{\\text\{max\}\}\\frac\{\\lVert vS\_\{p\}\\rVert\}\{\\lVert v\\rVert\}=\\underset\{\\hat\{v\}\\neq 0\}\{\\text\{max\}\}\\frac\{\\lVert\\hat\{v\}S\_\{p\}^\{\-1\}S\_\{p\}\\rVert\}\{\\lVert\\hat\{v\}S\_\{p\}^\{\-1\}\\rVert\}=\\underset\{\\hat\{v\}\\neq 0\}\{\\text\{max\}\}\\frac\{\\lVert\\hat\{v\}\\rVert\}\{\\lVert\\hat\{v\}S\_\{p\}^\{\-1\}\\rVert\}=\\frac\{1\}\{\\underset\{\\hat\{v\}\\neq 0\}\{\\text\{min\}\}\\frac\{\\lVert\\hat\{v\}S\_\{p\}^\{\-1\}\\rVert\}\{\\lVert\\hat\{v\}\\rVert\}\}=\\frac\{1\}\{\\underset\{\\lVert\\hat\{v\}\\rVert=1\}\{\\text\{min\}\}\\lVert\\hat\{v\}S\_\{p\}^\{\-1\}\\rVert\}\\leq\\frac\{1\}\{\\tau\_\{p\}\}\.
Next, we prove that for any vectorv≠0v\\neq 0,v​Sp​vT≥mp​∥v∥2vS\_\{p\}v^\{T\}\\geq m\_\{p\}\\lVert v\\rVert^\{2\}\. SinceS​TpST\_\{p\}is real symmetric matrix, denote its orthogonal diagonalization asS​Tp=Q​Λ​QTST\_\{p\}=Q\\Lambda Q^\{T\}andQ​QT=QT​Q=IQQ^\{T\}=Q^\{T\}Q=I,Λ=diag​\(λ1,…,λn\)\\Lambda=\\text\{diag\}\(\\lambda\_\{1\},\.\.\.,\\lambda\_\{n\}\)is the diagonal matrix composed ofS​TpST\_\{p\}’s eigenvalues\. Therefore, for any vectorv≠0v\\neq 0, lety=v​Qy=vQand we have

v​Sp​vT=v​S​Tp​vT=v​Q​Λ​QT​vT=y​Λ​yT=∑i=1nλi​yi2\.vS\_\{p\}v^\{T\}=vST\_\{p\}v^\{T\}=vQ\\Lambda Q^\{T\}v^\{T\}=y\\Lambda y^\{T\}=\\sum\_\{i=1\}^\{n\}\\lambda\_\{i\}y\_\{i\}^\{2\}\.Notice thatλi≥mp\\lambda\_\{i\}\\geq m\_\{p\}, sov​Sp​vT=∑i=1nλi​yi2≥mp​∥y∥2=mp​∥v∥2vS\_\{p\}v^\{T\}=\\sum\_\{i=1\}^\{n\}\\lambda\_\{i\}y\_\{i\}^\{2\}\\geq m\_\{p\}\\lVert y\\rVert^\{2\}=m\_\{p\}\\lVert v\\rVert^\{2\}sinceQQis a orthogonal matrix\.

Finally, we prove the main conclusion\. By[Theorem4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5), we decompose vector𝔼Pℋ​\[−∇θℓθ\]\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]into the direction of vector∇θJp\\nabla\_\{\\theta\}J\_\{p\}and the direction perpendicular to it, and we can get

∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=11−λℋθ⟨∇θJp​\(θ∗\),Sp​𝔼Pℋ​\[−∇θℓθ​\(θ∗\)\]⟩\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\big<\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\ \\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\big\>\(33\)=11−λℋθ​\(ρp​∥𝔼Pℋ​\[−∇θℓθ\]∥∥∇θJp∥​∇θJp⋅Sp⋅∇θJpT⏟\(A\)\+1−ρp2​∥𝔼Pℋ​\[−∇θℓθ\]∥​∇θJp⋅Sp⋅wT⏟\(B\)\)\\displaystyle=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\Big\(\\underbrace\{\\frac\{\\rho\_\{p\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\}\{\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\}\\nabla\_\{\\theta\}J\_\{p\}\\cdot S\_\{p\}\\cdot\\nabla\_\{\\theta\}J\_\{p\}^\{T\}\}\_\{\(A\)\}\+\\underbrace\{\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\nabla\_\{\\theta\}J\_\{p\}\\cdot S\_\{p\}\\cdot w^\{T\}\}\_\{\(B\)\}\\Big\)wherew⟂∇θJp,∥w∥=1w\\perp\\nabla\_\{\\theta\}J\_\{p\},\\ \\lVert w\\rVert=1\. For the term\(A\)\(A\), notice that∥Sp∥≤1τp\\lVert S\_\{p\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{p\}\}and for anyv≠0v\\neq 0,v​Sp​vT≥mp​∥v∥2vS\_\{p\}v^\{T\}\\geq m\_\{p\}\\lVert v\\rVert^\{2\}, we have

ρp​∥𝔼Pℋ​\[−∇θℓθ\]∥∥∇θJp∥​∥∇θJp∥2​mp≤\(A\)≤ρp​∥𝔼Pℋ​\[−∇θℓθ\]∥∥∇θJp∥​∥∇θJp∥2​1τp,if​ρp\>0\\displaystyle\\frac\{\\rho\_\{p\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\}\{\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\}\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert^\{2\}m\_\{p\}\\leq\(A\)\\leq\\frac\{\\rho\_\{p\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\}\{\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\}\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert^\{2\}\\frac\{1\}\{\\tau\_\{p\}\},\\ \\text\{if\}\\ \\rho\_\{p\}\>0\(34\)ρp​∥𝔼Pℋ​\[−∇θℓθ\]∥∥∇θJp∥​∥∇θJp∥2​1τp≤\(A\)≤ρp​∥𝔼Pℋ​\[−∇θℓθ\]∥∥∇θJp∥​∥∇θJp∥2​mp,if​ρp≤0\\displaystyle\\frac\{\\rho\_\{p\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\}\{\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\}\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert^\{2\}\\frac\{1\}\{\\tau\_\{p\}\}\\leq\(A\)\\leq\\frac\{\\rho\_\{p\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\}\{\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\}\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert^\{2\}m\_\{p\},\\ \\text\{if\}\\ \\rho\_\{p\}\\leq 0For the term\(B\)\(B\), using∥Sp∥≤1τp\\lVert S\_\{p\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{p\}\}, it’s easy to verify that

−1−ρp2τp​∥𝔼Pℋ​\[−∇θℓθ\]∥​∥∇θJp∥≤\(B\)≤1−ρp2τp​∥𝔼Pℋ​\[−∇θℓθ\]∥​∥∇θJp∥\.\-\\frac\{\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\}\{\\tau\_\{p\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\\leq\(B\)\\leq\\frac\{\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\}\{\\tau\_\{p\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\.\(35\)Combining Eq\. \([34](https://arxiv.org/html/2605.29267#A6.E34)\) and Eq\. \([35](https://arxiv.org/html/2605.29267#A6.E35)\) into Eq\. \([33](https://arxiv.org/html/2605.29267#A6.E33)\), we can get

11−λℋθ∥𝔼Pℋ\[∇θℓθ\]∥∥∇θJp∥\(\\displaystyle\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\\big\(ρpmp−1−ρp2τp\)≤∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\rho\_\{p\}m\_\{p\}\-\\frac\{\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\}\{\\tau\_\{p\}\}\\big\)\\leq\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\(36\)≤11−λℋθ​∥𝔼Pℋ​\[∇θℓθ\]∥​∥∇θJp∥​\(ρp\+1−ρp2τp\),if​ρp\>0\\displaystyle\\leq\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\\big\(\\frac\{\\rho\_\{p\}\+\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\}\{\\tau\_\{p\}\}\\big\),\\ \\text\{if\}\\ \\rho\_\{p\}\>011−λℋθ∥𝔼Pℋ\[∇θℓθ\]∥∥∇θJp∥\(\\displaystyle\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\\big\(ρp−1−ρp2τp\)≤∂Jp​\(θ∗\)∂λℋθ\\displaystyle\\frac\{\\rho\_\{p\}\-\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\}\{\\tau\_\{p\}\}\\big\)\\leq\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}≤11−λℋθ​∥𝔼Pℋ​\[∇θℓθ\]∥​∥∇θJp∥​\(ρp​mp\+1−ρp2τp\),if​ρp≤0\\displaystyle\\leq\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\\big\(\\rho\_\{p\}m\_\{p\}\+\\frac\{\\sqrt\{1\-\\rho\_\{p\}^\{2\}\}\}\{\\tau\_\{p\}\}\\big\),\\ \\text\{if\}\\ \\rho\_\{p\}\\leq 0Notice that11−λℋθ​∥𝔼Pℋ​\[∇θℓθ\]∥​∥∇θJp∥\>0\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\lVert\\mathbb\{E\}\_\{P\_\{\\mathcal\{H\}\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\]\\rVert\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\>0, so based on Eq\. \([36](https://arxiv.org/html/2605.29267#A6.E36)\) whenρp\>0\\rho\_\{p\}\>0, ifρp\>11\+mp2​τp2\\rho\_\{p\}\>\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\}then∂Jp​\(θ∗\)∂λℋθ\>0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0; whenρp≤0\\rho\_\{p\}\\leq 0, if−ρp\>11\+mp2​τp2\-\\rho\_\{p\}\>\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\}then∂Jp​\(θ∗\)∂λℋθ<0\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}<0i\.e\.sign⁡\(ρp\)​∂Jp​\(θ∗\)∂λℋθ\>0\\operatorname\{sign\}\(\\rho\_\{p\}\)\\frac\{\\partial J\_\{p\}\(\\theta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\>0\. ∎

### F\.7Proof of Corollary[E\.2](https://arxiv.org/html/2605.29267#A5.Thmtheorem2)

Corollary[E\.2](https://arxiv.org/html/2605.29267#A5.Thmtheorem2)\.Briefly noteτq=γϕ−Lϕ​εϕ−Lθ​εθ​Lϕ​εϕγθ−Lθ​εθ∈\(0,1\)\\tau\_\{q\}=\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\}\\in\(0,1\)and letmqm\_\{q\}be the minimal eigenvalue ofSq\+SqT2\\frac\{S\_\{q\}\+S\_\{q\}^\{T\}\}\{2\}, we have∥Sq∥≤1τq\\lVert S\_\{q\}\\rVert\\leq\\frac\{1\}\{\\tau\_\{q\}\}and if\|ρq\|\>11\+mp2​τp2\|\\rho\_\{q\}\|\>\\frac\{1\}\{\\sqrt\{1\+m\_\{p\}^\{2\}\\tau\_\{p\}^\{2\}\}\}, thensign⁡\(ρq\)⋅∂Jq​\(β∗\)∂λℋθ<0\\operatorname\{sign\}\(\\rho\_\{q\}\)\\cdot\\frac\{\\partial J\_\{q\}\(\\beta^\{\*\}\)\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}<0\.

###### Lemma F\.4\.

\(Two iteration paths’ distribution distance bounds between adjacent rounds\) Suppose Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)and[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)hold, and data spaces𝒳\\mathcal\{X\},𝒴\\mathcal\{Y\}are bounded\. There existsKc\>0K\_\{c\}\>0is a positive such that for any∀\(θ1,ϕ1\),\(θ2,ϕ2\)\\forall\\ \(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\), any mixing weightsλℛj,λℋj,λ𝒮j\\lambda\_\{\\mathcal\{R\}\}^\{j\},\\lambda\_\{\\mathcal\{H\}\}^\{j\},\\lambda\_\{\\mathcal\{S\}\}^\{j\},j∈\{θ,ϕ\}j\\in\\\{\\theta,\\phi\\\}, and any cross model data fractionsλθϕ,λϕθ\\lambda\_\{\\theta\}^\{\\phi\},\\lambda\_\{\\phi\}^\{\\theta\},

W\(P\(θ1,ϕ1\),\\displaystyle W\(P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(θ2,ϕ2\)\)≤\(1−λℛθ\)\[\(1−λθϕ\)Kc∥θ1−θ2∥\+λθϕKc∥ϕ1−ϕ2∥\\displaystyle P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\Big\[\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)K\_\{c\}\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\+\\lambda\_\{\\theta\}^\{\\phi\}K\_\{c\}\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\+\\displaystyle\+\(1−λθϕ\)\(Kc\+1\)W\(P\(θ1−,ϕ1−\),P\(θ2−,ϕ2−\)\)\+λθϕ\(Kc\+1\)W\(Q\(θ1−,ϕ1−\),Q\(θ2−,ϕ2−\)\)\],\\displaystyle\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)W\(P\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),P\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)W\(Q\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),Q\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\\Big\],\(37\)W\(Q\(θ1,ϕ1\),\\displaystyle W\(Q\(\\theta\_\{1\},\\phi\_\{1\}\),Q\(θ2,ϕ2\)\)≤\(1−λℛϕ\)\[\(1−λϕθ\)Kc∥ϕ1−ϕ2∥\+λϕθKc∥θ1−θ2∥\\displaystyle Q\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\)\\Big\[\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)K\_\{c\}\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}K\_\{c\}\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\+\\displaystyle\+\(1−λϕθ\)\(Kc\+1\)W\(Q\(θ1−,ϕ1−\),Q\(θ2−,ϕ2−\)\)\+λϕθ\(Kc\+1\)W\(P\(θ1−,ϕ1−\),P\(θ2−,ϕ2−\)\)\],\\displaystyle\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),Q\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),P\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\\Big\],\(38\)whereWWis the Wasserstein distance and\(θi−,ϕi−\),i∈\{1,2\}\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\),i\\in\\\{1,2\\\}are the previous round’s model parameters and\(θi,ϕi\)\(\\theta\_\{i\},\\phi\_\{i\}\)are obtained after updating the model parameters\(θi−,ϕi−\)\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\)based on Eq\. \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\) for one whole round no matter the updates in this round are synchronous or asynchronous\.

###### Proof\.

Let𝒮p​\(θ^,ϕ^\),𝒮q​\(θ^,ϕ^\)\\mathcal\{S\}\_\{p\}\(\\hat\{\\theta\},\\hat\{\\phi\}\),\\mathcal\{S\}\_\{q\}\(\\hat\{\\theta\},\\hat\{\\phi\}\)be the distributions of modelθ\\theta’s and modelϕ\\phi’s synthetic model\-generated data with model parametersθ^\\hat\{\\theta\}andϕ^\\hat\{\\phi\}, respectively\. Similarly we denoteℋp​\(θ^,ϕ^\),ℋq​\(θ^,ϕ^\)\\mathcal\{H\}\_\{p\}\(\\hat\{\\theta\},\\hat\{\\phi\}\),\\mathcal\{H\}\_\{q\}\(\\hat\{\\theta\},\\hat\{\\phi\}\)as the distributions of modelθ\\theta’s and modelϕ\\phi’s curated synthetic data with model parametersθ^\\hat\{\\theta\}andϕ^\\hat\{\\phi\}, respectively\. Letℛp\\mathcal\{R\}\_\{p\}andℛq\\mathcal\{R\}\_\{q\}be the fixed real data distribution of modelθ\\thetaand modelϕ\\phi, respectively\. By the definition of the mixture distributions, we have

P​\(θ^,ϕ^\)=λ𝒮θ​𝒮p​\(θ^,ϕ^\)\+λℋθ​ℋp​\(θ^,ϕ^\)\+λℛθ​ℛp,Q​\(θ^,ϕ^\)=λ𝒮ϕ​𝒮q​\(θ^,ϕ^\)\+λℋϕ​ℋq​\(θ^,ϕ^\)\+λℛϕ​ℛq\.P\(\\hat\{\\theta\},\\hat\{\\phi\}\)=\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\\mathcal\{S\}\_\{p\}\(\\hat\{\\theta\},\\hat\{\\phi\}\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\\mathcal\{H\}\_\{p\}\(\\hat\{\\theta\},\\hat\{\\phi\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\\mathcal\{R\}\_\{p\},\\ Q\(\\hat\{\\theta\},\\hat\{\\phi\}\)=\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\}\\mathcal\{S\}\_\{q\}\(\\hat\{\\theta\},\\hat\{\\phi\}\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\\mathcal\{H\}\_\{q\}\(\\hat\{\\theta\},\\hat\{\\phi\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\\mathcal\{R\}\_\{q\}\.\(39\)
First, we prove that for synthetic data distribution,∀\(θ1,ϕ1\),\(θ2,ϕ2\)\\forall\\ \(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\),

W\(𝒮p\(θ1,ϕ1\),\\displaystyle W\(\\mathcal\{S\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),𝒮p\(θ2,ϕ2\)\)≤\(1−λθϕ\)L∥θ1−θ2∥\+λθϕL∥ϕ1−ϕ2∥\\displaystyle\\mathcal\{S\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)L\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\+\\lambda\_\{\\theta\}^\{\\phi\}L\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\(40\)\+\(1−λθϕ\)​\(L\+1\)​W​\(P​\(θ1−,ϕ1−\),P​\(θ2−,ϕ2−\)\)\+λθϕ​\(L\+1\)​W​\(Q​\(θ1−,ϕ1−\),Q​\(θ2−,ϕ2−\)\),\\displaystyle\+\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(L\+1\)W\(P\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),P\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(L\+1\)W\(Q\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),Q\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\),where\(θi−,ϕi−\)\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\)are the parameters at the previous round andLLis the Lipschitz constant for models and reward functions\. By the definition of𝒮p\\mathcal\{S\}\_\{p\}and Eq\. \([1](https://arxiv.org/html/2605.29267#S2.E1)\)\-\([2](https://arxiv.org/html/2605.29267#S2.E2)\), for any random variablesz1∼𝒮p​\(θ1,ϕ1\),z2∼𝒮p​\(θ2,ϕ2\)z\_\{1\}\\sim\\mathcal\{S\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),z\_\{2\}\\sim\\mathcal\{S\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)we can get

z1=\(1−λθϕ\)​\(xp1,yp1\)\+λθϕ​\(xq1,yq1\),z2=\(1−λθϕ\)​\(xp2,yp2\)\+λθϕ​\(xq2,yq2\)\.\\displaystyle z\_\{1\}=\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(x\_\{p\}^\{1\},y\_\{p\}^\{1\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(x\_\{q\}^\{1\},y\_\{q\}^\{1\}\),z\_\{2\}=\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(x\_\{p\}^\{2\},y\_\{p\}^\{2\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(x\_\{q\}^\{2\},y\_\{q\}^\{2\}\)\.\(41\)wherexp1=p​\(yp1,θ1\),yp1∼Py​\(θ1−,ϕ1−\),yq1=q​\(xq1,ϕ1\),xq1∼Qx​\(θ1−,ϕ1−\)x\_\{p\}^\{1\}=p\(y\_\{p\}^\{1\},\\theta\_\{1\}\),y\_\{p\}^\{1\}\\sim P^\{y\}\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),y\_\{q\}^\{1\}=q\(x\_\{q\}^\{1\},\\phi\_\{1\}\),x\_\{q\}^\{1\}\\sim Q^\{x\}\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\)andxp2∼p​\(yp2,θ2\),yp2∼Py​\(θ2−,ϕ2−\),yq2∼q​\(xq2,ϕ1\),xq2∼Qx​\(θ2−,ϕ2−\)x\_\{p\}^\{2\}\\sim p\(y\_\{p\}^\{2\},\\theta\_\{2\}\),y\_\{p\}^\{2\}\\sim P^\{y\}\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\),y\_\{q\}^\{2\}\\sim q\(x\_\{q\}^\{2\},\\phi\_\{1\}\),x\_\{q\}^\{2\}\\sim Q^\{x\}\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\. Therefore,

W​\(𝒮p​\(θ1,ϕ1\),𝒮p​\(θ2,ϕ2\)\)=infπ∈Π​\(𝒮p​\(θ1,ϕ1\),𝒮p​\(θ2,ϕ2\)\)​𝔼\(ω1,ω2\)∼π​∥ω1−ω2∥≤𝔼​∥z1−z2∥\\displaystyle W\(\\mathcal\{S\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{S\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)=\\underset\{\\pi\\in\\Pi\(\\mathcal\{S\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{S\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\}\{\\text\{inf\}\}\\mathbb\{E\}\_\{\(\\omega\_\{1\},\\omega\_\{2\}\)\\sim\\pi\}\\lVert\\omega\_\{1\}\-\\omega\_\{2\}\\rVert\\leq\\mathbb\{E\}\\lVert z\_\{1\}\-z\_\{2\}\\rVert\(42\)=\\displaystyle=𝔼​∥\(1−λθϕ\)​\(\(p​\(yp1,θ1\),yp1\)−\(p​\(yp2,θ2\),yp2\)\)\+λθϕ​\(\(xq1,q​\(xq1,ϕ1\)\)−\(xq2,q​\(xq2,ϕ2\)\)\)∥\\displaystyle\\ \\mathbb\{E\}\\big\\lVert\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\\big\(\(p\(y\_\{p\}^\{1\},\\theta\_\{1\}\),y\_\{p\}^\{1\}\)\-\(p\(y\_\{p\}^\{2\},\\theta\_\{2\}\),y\_\{p\}^\{2\}\)\\big\)\+\\lambda\_\{\\theta\}^\{\\phi\}\\big\(\(x\_\{q\}^\{1\},q\(x\_\{q\}^\{1\},\\phi\_\{1\}\)\)\-\(x\_\{q\}^\{2\},q\(x\_\{q\}^\{2\},\\phi\_\{2\}\)\)\\big\)\\big\\rVert≤\\displaystyle\\leq\(1−λθϕ\)​\(𝔼​∥p​\(yp1,θ1\)−p​\(yp2,θ2\)∥\+𝔼​∥yp1−yp2∥\)\+λθϕ​\(𝔼​∥q​\(xq1,ϕ1\)−q​\(xq2,ϕ2\)∥\+𝔼​∥xq1−xq2∥\)\\displaystyle\\ \(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\\big\(\\mathbb\{E\}\\lVert p\(y\_\{p\}^\{1\},\\theta\_\{1\}\)\-p\(y\_\{p\}^\{2\},\\theta\_\{2\}\)\\rVert\+\\mathbb\{E\}\\lVert y\_\{p\}^\{1\}\-y\_\{p\}^\{2\}\\rVert\\big\)\+\\lambda\_\{\\theta\}^\{\\phi\}\\big\(\\mathbb\{E\}\\lVert q\(x\_\{q\}^\{1\},\\phi\_\{1\}\)\-q\(x\_\{q\}^\{2\},\\phi\_\{2\}\)\\rVert\+\\mathbb\{E\}\\lVert x\_\{q\}^\{1\}\-x\_\{q\}^\{2\}\\rVert\\big\)≤\\displaystyle\\leq\(1−λθϕ\)​\(L​∥θ1−θ2∥\+\(L\+1\)​𝔼​∥yp1−yp2∥\)\+λθϕ​\(L​∥ϕ1−ϕ2∥\+\(L\+1\)​𝔼​∥xq1−xq2∥\)\.\\displaystyle\\ \(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\\big\(L\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\+\(L\+1\)\\mathbb\{E\}\\lVert y\_\{p\}^\{1\}\-y\_\{p\}^\{2\}\\rVert\\big\)\+\\lambda\_\{\\theta\}^\{\\phi\}\\big\(L\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\+\(L\+1\)\\mathbb\{E\}\\lVert x\_\{q\}^\{1\}\-x\_\{q\}^\{2\}\\rVert\\big\)\.The last step is derived from the Lipschitz property of modelsθ,ϕ\\theta,\\phi, and under∥∥\\lVert\\rVertnorm, marginal distribution’sWWdistance is less than theWWdistance ofPP\. Since Eq\. \([42](https://arxiv.org/html/2605.29267#A6.E42)\) is satisfied for anyz1,z2z\_\{1\},z\_\{2\}, taking inf on both sides and we can get Eq\. \([40](https://arxiv.org/html/2605.29267#A6.E40)\)\. Similarly, we can prove the same conclusion for𝒮q\\mathcal\{S\}\_\{q\}\.

Next, we prove similar results for synthetic curation data distributions\. By Eq\. \([3](https://arxiv.org/html/2605.29267#S2.E3)\), for the modelθ\\theta,\(x1,\.\.,xK\)\(x\_\{1\},\.\.,x\_\{K\}\)sampled independently fromp​\(y,θ\)p\(y,\\theta\), then user picks one samplex^\\hat\{x\}based on the following probabilities:

ℙ\(x^=xk\|x1,…,xK,y\)=er​\(y,xk\)∑i=1Ker​\(y,xi\),denote asx^∼ℬ𝒯\(x1,\.\.,xK\)\.\\mathbb\{P\}\(\\hat\{x\}=x\_\{k\}\|x\_\{1\},\.\.\.,x\_\{K\},y\)=\\frac\{e^\{r\(y,x\_\{k\}\)\}\}\{\\sum\_\{i=1\}^\{K\}e^\{r\(y,x\_\{i\}\)\}\},\\ \\ \\text\{denote as \}\\hat\{x\}\\sim\\mathcal\{BT\}\(x\_\{1\},\.\.,x\_\{K\}\)\.\(43\)LetKc=L​K​\(1\+L​B2\)\>0K\_\{c\}=LK\(1\+\\frac\{LB\}\{2\}\)\>0be a constant whereLLis the Lipschitz constant for model functions and reward,BBis the data norm’s upper bound \(𝒳,𝒴\\mathcal\{X\},\\mathcal\{Y\}are bounded by assumption\) andKKis the user curation sampling number\. We prove that for∀\(θ1,ϕ1\),\(θ2,ϕ2\)\\forall\\ \(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\)

W\(ℋp\(θ1,ϕ1\),\\displaystyle W\(\\mathcal\{H\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),ℋp\(θ2,ϕ2\)\)≤\(1−λθϕ\)Kc∥θ1−θ2∥\+λθϕKc∥ϕ1−ϕ2∥\\displaystyle\\mathcal\{H\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)K\_\{c\}\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\+\\lambda\_\{\\theta\}^\{\\phi\}K\_\{c\}\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\(44\)\+\(1−λθϕ\)​\(Kc\+1\)​W​\(P​\(θ1−,ϕ1−\),P​\(θ2−,ϕ2−\)\)\+λθϕ​\(Kc\+1\)​W​\(Q​\(θ1−,ϕ1−\),Q​\(θ2−,ϕ2−\)\),\\displaystyle\+\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)W\(P\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),P\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)W\(Q\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),Q\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\),where\(θi−,ϕi−\)\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\)are the parameters for previous mixture distributions\. By the definition ofℋp\\mathcal\{H\}\_\{p\}and combined with notations in Eq\. \([41](https://arxiv.org/html/2605.29267#A6.E41)\), for any random variablesz1c∼ℋp​\(θ1,ϕ1\)z\_\{1\}^\{c\}\\sim\\mathcal\{H\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),z2c∼ℋp​\(θ2,ϕ2\)z\_\{2\}^\{c\}\\sim\\mathcal\{H\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\),

z1c=\(1−λθϕ\)​\(x^p1,yp1\)\+λθϕ​\(xq1,y^q1\),z2c=\(1−λθϕ\)​\(x^p2,yp2\)\+λθϕ​\(xq2,y^q2\),z\_\{1\}^\{c\}=\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(\\hat\{x\}\_\{p\}^\{1\},y\_\{p\}^\{1\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(x\_\{q\}^\{1\},\\hat\{y\}\_\{q\}^\{1\}\),z\_\{2\}^\{c\}=\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(\\hat\{x\}\_\{p\}^\{2\},y\_\{p\}^\{2\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(x\_\{q\}^\{2\},\\hat\{y\}\_\{q\}^\{2\}\),\(45\)wherex^p1∼ℬ​𝒯​\(xp1​\(1\),…,xp1​\(K\)\),xp1​\(i\)​∼i\.i\.d\.​p​\(yp1,θ1\),yp1∼Py​\(θ1−,ϕ1−\),y^q1∼ℬ​𝒯​\(yq1​\(1\),…​yq1​\(K\)\),yq1​\(i\)​∼i\.i\.d\.​q​\(xq1,ϕ1\),xq1∼Qx​\(θ1−,ϕ1−\)\\hat\{x\}\_\{p\}^\{1\}\\sim\\mathcal\{BT\}\(x\_\{p\}^\{1\}\(1\),\.\.\.,x\_\{p\}^\{1\}\(K\)\),x\_\{p\}^\{1\}\(i\)\\overset\{i\.i\.d\.\}\{\\sim\}p\(y\_\{p\}^\{1\},\\theta\_\{1\}\),y\_\{p\}^\{1\}\\sim P^\{y\}\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),\\hat\{y\}\_\{q\}^\{1\}\\sim\\mathcal\{BT\}\(y\_\{q\}^\{1\}\(1\),\.\.\.y\_\{q\}^\{1\}\(K\)\),y\_\{q\}^\{1\}\(i\)\\overset\{i\.i\.d\.\}\{\\sim\}q\(x\_\{q\}^\{1\},\\phi\_\{1\}\),x\_\{q\}^\{1\}\\sim Q^\{x\}\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\)andx^p2∼ℬ​𝒯​\(xp2​\(1\),…,xp2​\(K\)\),xp2​\(i\)​∼i\.i\.d\.​p​\(yp2,θ2\),yp2∼Py​\(θ2−,ϕ2−\),y^q2∼ℬ​𝒯​\(yq2​\(1\),…​yq2​\(K\)\),yq2​\(i\)​∼i\.i\.d\.​q​\(xq2,ϕ2\),xq2∼Qx​\(θ2−,ϕ2−\)\\hat\{x\}\_\{p\}^\{2\}\\sim\\mathcal\{BT\}\(x\_\{p\}^\{2\}\(1\),\.\.\.,x\_\{p\}^\{2\}\(K\)\),x\_\{p\}^\{2\}\(i\)\\overset\{i\.i\.d\.\}\{\\sim\}p\(y\_\{p\}^\{2\},\\theta\_\{2\}\),y\_\{p\}^\{2\}\\sim P^\{y\}\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\),\\hat\{y\}\_\{q\}^\{2\}\\sim\\mathcal\{BT\}\(y\_\{q\}^\{2\}\(1\),\.\.\.y\_\{q\}^\{2\}\(K\)\),y\_\{q\}^\{2\}\(i\)\\overset\{i\.i\.d\.\}\{\\sim\}q\(x\_\{q\}^\{2\},\\phi\_\{2\}\),x\_\{q\}^\{2\}\\sim Q^\{x\}\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\. Therefore, similar to Eq\. \([42](https://arxiv.org/html/2605.29267#A6.E42)\), we have

W​\(ℋp​\(θ1,ϕ1\),ℋp​\(θ2,ϕ2\)\)≤\(1−λθϕ\)​\(𝔼​∥x^p1−x^p2∥\+𝔼​∥yp1−yp2∥\)\+λθϕ​\(𝔼​∥y^q1−y^q2∥\+𝔼​∥xq1−xq2∥\)\.W\(\\mathcal\{H\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{H\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\\big\(\\mathbb\{E\}\\lVert\\hat\{x\}\_\{p\}^\{1\}\-\\hat\{x\}\_\{p\}^\{2\}\\rVert\+\\mathbb\{E\}\\lVert y\_\{p\}^\{1\}\-y\_\{p\}^\{2\}\\rVert\\big\)\+\\lambda\_\{\\theta\}^\{\\phi\}\\big\(\\mathbb\{E\}\\lVert\\hat\{y\}\_\{q\}^\{1\}\-\\hat\{y\}\_\{q\}^\{2\}\\rVert\+\\mathbb\{E\}\\lVert x\_\{q\}^\{1\}\-x\_\{q\}^\{2\}\\rVert\\big\)\.\(46\)For the term𝔼​∥x^p1−x^p2∥\\mathbb\{E\}\\lVert\\hat\{x\}\_\{p\}^\{1\}\-\\hat\{x\}\_\{p\}^\{2\}\\rVert, by its definition,

𝔼∥x^p1−x^p2∥=𝔼∥∑i=1Kpi1xp1\(i\)−∑i=1Kpi2xp2\(i\)∥,pik=er​\(xpk​\(i\)\)∑j=1Ker​\(xpk​\(j\)\),i∈\{1,\.\.,K\},k∈\{1,2\}\\displaystyle\\mathbb\{E\}\\lVert\\hat\{x\}\_\{p\}^\{1\}\-\\hat\{x\}\_\{p\}^\{2\}\\rVert=\\mathbb\{E\}\\lVert\\sum\_\{i=1\}^\{K\}p\_\{i\}^\{1\}x\_\{p\}^\{1\}\(i\)\-\\sum\_\{i=1\}^\{K\}p\_\{i\}^\{2\}x\_\{p\}^\{2\}\(i\)\\rVert,\\ p\_\{i\}^\{k\}=\\frac\{e^\{r\(x\_\{p\}^\{k\}\(i\)\)\}\}\{\\sum\_\{j=1\}^\{K\}e^\{r\(x\_\{p\}^\{k\}\(j\)\)\}\},\\ i\\in\\\{1,\.\.,K\\\},k\\in\\\{1,2\\\}\(47\)=\\displaystyle=𝔼​∥∑i=1Kpi1​\(xp1​\(i\)−xp2​\(i\)\)−∑i=1K\(pi1−pi2\)​xp2​\(i\)∥≤𝔼​∑i=1Kpi1​∥xp1​\(i\)−xp2​\(i\)∥\+B​∑i=1K𝔼​\|pi1−pi2\|\\displaystyle\\mathbb\{E\}\\lVert\\sum\_\{i=1\}^\{K\}p\_\{i\}^\{1\}\(x\_\{p\}^\{1\}\(i\)\-x\_\{p\}^\{2\}\(i\)\)\-\\sum\_\{i=1\}^\{K\}\(p\_\{i\}^\{1\}\-p\_\{i\}^\{2\}\)x\_\{p\}^\{2\}\(i\)\\rVert\\leq\\mathbb\{E\}\\sum\_\{i=1\}^\{K\}p\_\{i\}^\{1\}\\lVert x\_\{p\}^\{1\}\(i\)\-x\_\{p\}^\{2\}\(i\)\\rVert\+B\\sum\_\{i=1\}^\{K\}\\mathbb\{E\}\|p\_\{i\}^\{1\}\-p\_\{i\}^\{2\}\|=\\displaystyle=𝔼​∑i=1Kpi1​∥p​\(yp1,θ1\)−p​\(yp2,θ2\)∥\+B​∑i=1K𝔼​\|pi1−pi2\|≤L​K​\(𝔼​∥yp1−yp2∥\+∥θ1−θ2∥\)\+B​∑i=1K𝔼​\|pi1−pi2\|\.\\displaystyle\\mathbb\{E\}\\sum\_\{i=1\}^\{K\}p\_\{i\}^\{1\}\\lVert p\(y\_\{p\}^\{1\},\\theta\_\{1\}\)\-p\(y\_\{p\}^\{2\},\\theta\_\{2\}\)\\rVert\+B\\sum\_\{i=1\}^\{K\}\\mathbb\{E\}\|p\_\{i\}^\{1\}\-p\_\{i\}^\{2\}\|\\leq LK\\big\(\\mathbb\{E\}\\lVert y\_\{p\}^\{1\}\-y\_\{p\}^\{2\}\\rVert\+\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\\big\)\+B\\sum\_\{i=1\}^\{K\}\\mathbb\{E\}\|p\_\{i\}^\{1\}\-p\_\{i\}^\{2\}\|\.For the softmax function, it’s easy to check that its 0\.5\-Lipschitz under theℓ1\\ell\_\{1\}norm, and sincerris also Lipschitz,

B​∑i=1K𝔼​\|pi1−pi2\|\\displaystyle B\\sum\_\{i=1\}^\{K\}\\mathbb\{E\}\|p\_\{i\}^\{1\}\-p\_\{i\}^\{2\}\|≤B2​∑i=1K𝔼​\|r​\(xp1​\(i\)\)−r​\(xp2​\(i\)\)\|≤L​B2​∑i=1K𝔼​∥xp1​\(i\)−xp2​\(i\)∥\\displaystyle\\leq\\frac\{B\}\{2\}\\sum\_\{i=1\}^\{K\}\\mathbb\{E\}\|r\(x\_\{p\}^\{1\}\(i\)\)\-r\(x\_\{p\}^\{2\}\(i\)\)\|\\leq\\frac\{LB\}\{2\}\\sum\_\{i=1\}^\{K\}\\mathbb\{E\}\\lVert x\_\{p\}^\{1\}\(i\)\-x\_\{p\}^\{2\}\(i\)\\rVert\(48\)≤L2​B​K2​\(𝔼​∥yp1−yp2∥\+∥θ1−θ2∥\)\.\\displaystyle\\leq\\frac\{L^\{2\}BK\}\{2\}\\big\(\\mathbb\{E\}\\lVert y\_\{p\}^\{1\}\-y\_\{p\}^\{2\}\\rVert\+\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\\big\)\.Combine Eq\. \([47](https://arxiv.org/html/2605.29267#A6.E47)\) and Eq\. \([48](https://arxiv.org/html/2605.29267#A6.E48)\), we have

𝔼​∥x^p1−x^p2∥≤L​K​\(1\+L​B2\)​\(𝔼​∥yp1−yp2∥\+∥θ1−θ2∥\)\.\\mathbb\{E\}\\lVert\\hat\{x\}\_\{p\}^\{1\}\-\\hat\{x\}\_\{p\}^\{2\}\\rVert\\leq LK\(1\+\\frac\{LB\}\{2\}\)\\big\(\\mathbb\{E\}\\lVert y\_\{p\}^\{1\}\-y\_\{p\}^\{2\}\\rVert\+\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert\\big\)\.\(49\)Similarly, we can get the upper bound of𝔼​∥y^q1−y^q2∥\\mathbb\{E\}\\lVert\\hat\{y\}\_\{q\}^\{1\}\-\\hat\{y\}\_\{q\}^\{2\}\\rVertas following

𝔼​∥y^q1−y^q2∥≤L​K​\(1\+L​B2\)​\(𝔼​∥xq1−xq2∥\+∥ϕ1−ϕ2∥\)\.\\mathbb\{E\}\\lVert\\hat\{y\}\_\{q\}^\{1\}\-\\hat\{y\}\_\{q\}^\{2\}\\rVert\\leq LK\(1\+\\frac\{LB\}\{2\}\)\\big\(\\mathbb\{E\}\\lVert x\_\{q\}^\{1\}\-x\_\{q\}^\{2\}\\rVert\+\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\\big\)\.\(50\)Substitute Eq\. \([49](https://arxiv.org/html/2605.29267#A6.E49)\) and Eq\. \([50](https://arxiv.org/html/2605.29267#A6.E50)\) into Eq\. \([46](https://arxiv.org/html/2605.29267#A6.E46)\) and notice the inequality between marginal distributionWWdistance andPP’sWWdistance, we can get Eq\. \([44](https://arxiv.org/html/2605.29267#A6.E44)\)\. Similarly, we can prove the same conclusion forℋq\\mathcal\{H\}\_\{q\}\.

Finally, we prove the lemma conclusion\. Recall the definition of the mixture distributionPPin Eq\. \([39](https://arxiv.org/html/2605.29267#A6.E39)\), and similarly we have for∀\(θ1,ϕ1\),\(θ2,ϕ2\)\\forall\\ \(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\),

W​\(P​\(θ1,ϕ1\),P​\(θ2,ϕ2\)\)\\displaystyle W\(P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\)\)≤λ𝒮θ​W​\(𝒮p​\(θ1,ϕ1\),𝒮p​\(θ2,ϕ2\)\)\+λℋθ​W​\(ℋp​\(θ1,ϕ1\),ℋp​\(θ2,ϕ2\)\)\+λℛθ​W​\(ℛp,ℛp\)\\displaystyle\\leq\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}W\(\\mathcal\{S\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{S\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}W\(\\mathcal\{H\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{H\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}W\(\\mathcal\{R\}\_\{p\},\\mathcal\{R\}\_\{p\}\)=λ𝒮θ​W​\(𝒮p​\(θ1,ϕ1\),𝒮p​\(θ2,ϕ2\)\)\+λℋθ​W​\(ℋp​\(θ1,ϕ1\),ℋp​\(θ2,ϕ2\)\)\.\\displaystyle=\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}W\(\\mathcal\{S\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{S\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}W\(\\mathcal\{H\}\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\mathcal\{H\}\_\{p\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\.Substitute Eq\. \([40](https://arxiv.org/html/2605.29267#A6.E40)\) and Eq\. \([44](https://arxiv.org/html/2605.29267#A6.E44)\) into the above equation, and we can prove Eq\. \([37](https://arxiv.org/html/2605.29267#A6.E37)\)\. Similarly, Eq\. \([38](https://arxiv.org/html/2605.29267#A6.E38)\) is also satisfied\. ∎

###### Corollary F\.6\.

\(Two iteration paths’ distribution distance bounds for single model update in one round\) Suppose Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)and[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)hold, and data spaces𝒳\\mathcal\{X\},𝒴\\mathcal\{Y\}are bounded\.KcK\_\{c\}is the positive constant in Lemma[F\.4](https://arxiv.org/html/2605.29267#A6.Thmtheorem4)\. For any mixing weightsλℛj,λℋj,λ𝒮j\\lambda\_\{\\mathcal\{R\}\}^\{j\},\\lambda\_\{\\mathcal\{H\}\}^\{j\},\\lambda\_\{\\mathcal\{S\}\}^\{j\},j∈\{θ,ϕ\}j\\in\\\{\\theta,\\phi\\\}, and any cross model data fractionsλθϕ,λϕθ\\lambda\_\{\\theta\}^\{\\phi\},\\lambda\_\{\\phi\}^\{\\theta\}, and for∀\(θ1,ϕ1\),\(θ2,ϕ2\)\\forall\\ \(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\), if\(θi−,ϕi−\),i∈\{1,2\}\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\),i\\in\\\{1,2\\\}are the previous round’s model parameters before\(θi,ϕi\),i∈\{1,2\}\(\\theta\_\{i\},\\phi\_\{i\}\),i\\in\\\{1,2\\\}, then during the iteration process to get\(θi,ϕi\)\(\\theta\_\{i\},\\phi\_\{i\}\), regardless of the iteration order \(Eq\. \([21](https://arxiv.org/html/2605.29267#A6.E21)\)\-\([23](https://arxiv.org/html/2605.29267#A6.E23)\)\) in the current iteration, the distribution distance in this round’s process \(θ^i∈\{θi,θi−\},ϕ^i∈\{ϕi,ϕi−\},i∈\{1,2\}\\hat\{\\theta\}\_\{i\}\\in\\\{\\theta\_\{i\},\\theta\_\{i\}^\{\-\}\\\},\\hat\{\\phi\}\_\{i\}\\in\\\{\\phi\_\{i\},\\phi\_\{i\}^\{\-\}\\\},i\\in\\\{1,2\\\}\) has the following inequality:

W\(P\(θ^1,ϕ^1\),\\displaystyle W\(P\(\\hat\{\\theta\}\_\{1\},\\hat\{\\phi\}\_\{1\}\),P\(θ^2,ϕ^2\)\)≤\(1−λℛθ\)\[\(1−λθϕ\)Kc∥θ^1−θ^2∥\+λθϕKc∥ϕ^1−ϕ^2∥\\displaystyle P\(\\hat\{\\theta\}\_\{2\},\\hat\{\\phi\}\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\Big\[\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)K\_\{c\}\\lVert\\hat\{\\theta\}\_\{1\}\-\\hat\{\\theta\}\_\{2\}\\rVert\+\\lambda\_\{\\theta\}^\{\\phi\}K\_\{c\}\\lVert\\hat\{\\phi\}\_\{1\}\-\\hat\{\\phi\}\_\{2\}\\rVert\(51\)\+\\displaystyle\+\(1−λθϕ\)\(Kc\+1\)W\(P\(θ1−,ϕ1−\),P\(θ2−,ϕ2−\)\)\+λθϕ\(Kc\+1\)W\(Q\(θ1−,ϕ1−\),Q\(θ2−,ϕ2−\)\)\],\\displaystyle\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)W\(P\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),P\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)W\(Q\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),Q\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\\Big\],W\(Q\(θ^1,ϕ^1\),\\displaystyle W\(Q\(\\hat\{\\theta\}\_\{1\},\\hat\{\\phi\}\_\{1\}\),Q\(θ^2,ϕ^2\)\)≤\(1−λℛϕ\)\[\(1−λϕθ\)Kc∥ϕ^1−ϕ^2∥\+λϕθKc∥θ^1−θ^2∥\\displaystyle Q\(\\hat\{\\theta\}\_\{2\},\\hat\{\\phi\}\_\{2\}\)\)\\leq\(1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\)\\Big\[\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)K\_\{c\}\\lVert\\hat\{\\phi\}\_\{1\}\-\\hat\{\\phi\}\_\{2\}\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}K\_\{c\}\\lVert\\hat\{\\theta\}\_\{1\}\-\\hat\{\\theta\}\_\{2\}\\rVert\+\\displaystyle\+\(1−λϕθ\)\(Kc\+1\)W\(Q\(θ1−,ϕ1−\),Q\(θ2−,ϕ2−\)\)\+λϕθ\(Kc\+1\)W\(P\(θ1−,ϕ1−\),P\(θ2−,ϕ2−\)\)\]\.\\displaystyle\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),Q\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\(\\theta\_\{1\}^\{\-\},\\phi\_\{1\}^\{\-\}\),P\(\\theta\_\{2\}^\{\-\},\\phi\_\{2\}^\{\-\}\)\)\\Big\]\.

###### Lemma F\.8\.

If the real data is bounded and data spaces𝒳,𝒴\\mathcal\{X\},\\mathcal\{Y\}are bounded, then for∀t\>0\\forall\\ t\>0, the mixture distributionsP​\(θt,ϕt\)P\(\\theta\_\{t\},\\phi\_\{t\}\)andQ​\(θt,ϕt\)Q\(\\theta\_\{t\},\\phi\_\{t\}\)have finite first moments\.

###### Proof\.

If real data is bounded which means forZ∼RpZ\\sim R\_\{p\}orZ∼RqZ\\sim R\_\{q\}, then∥Z∥≤M\\lVert Z\\rVert\\leq Mfor someM\>0M\>0then𝔼​\[∥Z∥\]≤M\\mathbb\{E\}\[\\lVert Z\\rVert\]\\leq Mis finite, real data distributions have finite first moment\. We use the induction method to prove this lemma\. Assume thatP​\(θt−1,ϕt−1\)P\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)andQ​\(θt−1,ϕt−1\)Q\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)are distributions with finite first moment, then forZ−∼P​\(θt−1,ϕt−1\)Z^\{\-\}\\sim P\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\), there existsM−\>0M^\{\-\}\>0such that

𝔼​\[∥Z−∥\]≤M−\.\\mathbb\{E\}\[\\lVert Z^\{\-\}\\rVert\]\\leq M^\{\-\}\.Next, we prove thatP​\(θt,ϕt\)P\(\\theta\_\{t\},\\phi\_\{t\}\)andQ​\(θt,ϕt\)Q\(\\theta\_\{t\},\\phi\_\{t\}\)still have finite first moment\. TakeZ∼P​\(θt,ϕt\)Z\\sim P\(\\theta\_\{t\},\\phi\_\{t\}\)with synchronous updates as an example\. SinceP​\(θt,ϕt\)P\(\\theta\_\{t\},\\phi\_\{t\}\)is a mixture distribution, forys,yc​∼i\.i\.d\.​Py​\(θt−1,ϕt−1\)y\_\{s\},y\_\{c\}\\overset\{i\.i\.d\.\}\{\\sim\}P^\{y\}\(\\theta\_\{t\-1\},\\phi\_\{t\-1\}\)andx^c∼ℬ​𝒯​\(xc​\(1\),…,xc​\(K\)\)\\hat\{x\}\_\{c\}\\sim\\mathcal\{BT\}\(x\_\{c\}\(1\),\.\.\.,x\_\{c\}\(K\)\),xc​\(i\)​∼i\.i\.d\.​p​\(yc,θt\)x\_\{c\}\(i\)\\overset\{i\.i\.d\.\}\{\\sim\}p\(y\_\{c\},\\theta\_\{t\}\), we can get

Z\\displaystyle Z=λ𝒮θ​\(p​\(ys,θt\),ys\)\+λℋθ​\(x^c,yc\)\+λℛθ​Zr,\\displaystyle=\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\(p\(y\_\{s\},\\theta\_\{t\}\),y\_\{s\}\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\(\\hat\{x\}\_\{c\},y\_\{c\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}Z\_\{r\},\(52\)𝔼​\[∥Z∥\]\\displaystyle\\mathbb\{E\}\[\\lVert Z\\rVert\]≤λ𝒮θ​\(𝔼​\[∥p​\(ys,θt\)∥\]\+𝔼​\[∥ys∥\]\)\+λℋθ​\(𝔼​\[∥x^c∥\]\+𝔼​\[∥yc∥\]\)\+λℛθ​M\\displaystyle\\leq\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\(\\mathbb\{E\}\[\\lVert p\(y\_\{s\},\\theta\_\{t\}\)\\rVert\]\+\\mathbb\{E\}\[\\lVert y\_\{s\}\\rVert\]\)\+\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\(\\mathbb\{E\}\[\\lVert\\hat\{x\}\_\{c\}\\rVert\]\+\\mathbb\{E\}\[\\lVert y\_\{c\}\\rVert\]\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}M≤\(1−λℛθ\)​\(B\+M−\)\+λℛθ​M<∞,\\displaystyle\\leq\(1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\)\(B\+M^\{\-\}\)\+\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}M<\\infty,whereBBis the upper bound of data space\. Therefore,P​\(θt,βt\)P\(\\theta\_\{t\},\\beta\_\{t\}\)has finite first moment for anyt\>0t\>0\. Similarly, we can prove thatQ​\(θt,βt\)Q\(\\theta\_\{t\},\\beta\_\{t\}\)has finite first moment for anyt\>0t\>0\. If training updates are asynchronous, the proof is similar\. ∎

### F\.8Proof of Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4)

Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4)\.Suppose Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)and[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3)hold, and data spaces𝒳\\mathcal\{X\},𝒴\\mathcal\{Y\}are bounded\. If both models are Lipschitz in their inputs andθ,ϕ\\theta,\\phiand reward functions are Lipschitz in the inputs, then there existsτ∈\(0,1\)\\tau\\in\(0,1\)such that if the fraction of real data in each round of model training is sufficiently large, i\.e\.,min⁡\(λℛθ,λℛϕ\)\>τ\\min\\big\(\\lambda^\{\\theta\}\_\{\\mathcal\{R\}\},\\lambda^\{\\phi\}\_\{\\mathcal\{R\}\}\\big\)\>\\tau, then a unique stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)exists\. Moreover, the iterative training loop \([2](https://arxiv.org/html/2605.29267#S2.Ex2)\) will drive the multi\-model ecosystem\(θt,ϕt\)\(\\theta\_\{t\},\\phi\_\{t\}\)to converge to\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\), and the training data distributions will also converge, i\.e\.,limt→∞Ptx,y=P​\(θ∗,ϕ∗\)\\lim\_\{t\\to\\infty\}P\_\{t\}^\{x,y\}=P\(\\theta^\{\*\},\\phi^\{\*\}\),limt→∞Qtx,y=Q​\(θ∗,ϕ∗\)\\lim\_\{t\\to\\infty\}Q\_\{t\}^\{x,y\}=Q\(\\theta^\{\*\},\\phi^\{\*\}\)\.

###### Proof\.

We only need to prove that there exists data ratios such that the iteration processes in Eq\. \([21](https://arxiv.org/html/2605.29267#A6.E21)\)\-\([23](https://arxiv.org/html/2605.29267#A6.E23)\) are always compression mappings, thus combining the Banach convergence theorem to prove this proposition\.

Recall the notationsRp,RqR\_\{p\},R\_\{q\}in Eq\. \([16](https://arxiv.org/html/2605.29267#A6.E16)\)\. First, we prove that for any\(θ1,ϕ1\),\(θ2,ϕ2\)\(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\),

∥argmin𝜃​Rp​\(θ,P​\(θ1,ϕ1\)\)−argmin𝜃​Rp​\(θ,P​\(θ2,ϕ2\)\)∥≤Lθγθ​W​\(P​\(θ1,ϕ1\),P​\(θ2,ϕ2\)\)\.\\lVert\\underset\{\\theta\}\{\\text\{argmin\}\}\\ R\_\{p\}\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\)\-\\underset\{\\theta\}\{\\text\{argmin\}\}\\ R\_\{p\}\(\\theta,P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\rVert\\leq\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\.\(53\)Similar to the proof in Thm\.[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6),R​\(θ,P​\(θ1,ϕ1\)\)R\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\)isγθ\\gamma\_\{\\theta\}\-strongly convex inθ\\thetafor any fixed\(θ1,ϕ1\)\(\\theta\_\{1\},\\phi\_\{1\}\)and there always exists unique minimal pointsφ1,φ2\\varphi\_\{1\},\\varphi\_\{2\}so that∇θRp​\(θ,P​\(θ1,ϕ1\)\)\|θ=φ1=∇θRp​\(θ,P​\(θ2,ϕ2\)\)\|θ=φ2=0\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\big\|\_\{\\theta=\\varphi\_\{1\}\}=\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{2\},\\phi\_\{2\}\)\\big\)\\big\|\_\{\\theta=\\varphi\_\{2\}\}=0\. Therefore, we can get the same result in Thm\.[3\.6](https://arxiv.org/html/2605.29267#S3.Thmtheorem6),

∥\(φ1−φ2\)T​\(∇θRp​\(φ1,P​\(θ1,ϕ1\)\)−∇θRp​\(φ2,P​\(θ1,ϕ1\)\)\)∥≥γθ​∥φ1−φ2∥2\.\\big\\lVert\(\\varphi\_\{1\}\-\\varphi\_\{2\}\)^\{T\}\\Big\(\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{1\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\Big\)\\big\\rVert\\geq\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert^\{2\}\.Notice that∇θRp​\(θ,P​\(θ1,ϕ1\)\)\|θ=φ1=0\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\theta,P\(\\theta\_\{1\},\\phi\_\{1\}\)\\big\)\\big\|\_\{\\theta=\\varphi\_\{1\}\}=0, so

∥∇θRp​\(φ2,P​\(θ1,ϕ1\)\)∥≥γθ​∥φ1−φ2∥\.\\lVert\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\)\\rVert\\geq\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert\.\(54\)Moreover, letΠ​\(P​\(θ1,ϕ1\),P​\(θ2,ϕ2\)\)\\Pi\(P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\)\)be set consisting all the distributions whose marginal distributions areP​\(θ1,ϕ1\)P\(\\theta\_\{1\},\\phi\_\{1\}\)andP​\(θ2,ϕ2\)P\(\\theta\_\{2\},\\phi\_\{2\}\)respectively\. For∀\(Z1,Z2\)∼π∈Π​\(P​\(θ1,ϕ1\),P​\(θ2,ϕ2\)\)\\forall\(Z\_\{1\},Z\_\{2\}\)\\sim\\pi\\in\\Pi\(P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\)\),

∥∇θRp​\(φ2,P​\(θ1,ϕ1\)\)−∇θRp​\(φ2,P​\(θ2,ϕ2\)\)∥\\displaystyle\\lVert\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\rVert=∥𝔼P​\(θ1,ϕ1\)​∇θℓθ​\(z;φ2\)−𝔼P​\(θ2,ϕ2\)​∇θℓθ​\(z;φ2\)∥\\displaystyle=\\lVert\\mathbb\{E\}\_\{P\(\\theta\_\{1\},\\phi\_\{1\}\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\varphi\_\{2\}\)\-\\mathbb\{E\}\_\{P\(\\theta\_\{2\},\\phi\_\{2\}\)\}\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(z;\\varphi\_\{2\}\)\\rVert\(55\)=∥𝔼\(Z1,Z2\)∼π​\[∇θℓθ​\(Z1;φ2\)−∇θℓθ​\(Z2;φ2\)\]∥\\displaystyle=\\lVert\\mathbb\{E\}\_\{\(Z\_\{1\},Z\_\{2\}\)\\sim\\pi\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(Z\_\{1\};\\varphi\_\{2\}\)\-\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(Z\_\{2\};\\varphi\_\{2\}\)\]\\rVert≤Lθ​𝔼\(Z1,Z2\)∼π​∥Z1−Z2∥,\\displaystyle\\leq L\_\{\\theta\}\\mathbb\{E\}\_\{\(Z\_\{1\},Z\_\{2\}\)\\sim\\pi\}\\lVert Z\_\{1\}\-Z\_\{2\}\\rVert,and sinceπ\\piis an arbitrary element inΠ\\Pi, taking inf on both sides of Eq\. \([55](https://arxiv.org/html/2605.29267#A6.E55)\) and combining Eq\. \([54](https://arxiv.org/html/2605.29267#A6.E54)\) we have

γθ​∥φ1−φ2∥\\displaystyle\\gamma\_\{\\theta\}\\lVert\\varphi\_\{1\}\-\\varphi\_\{2\}\\rVert≤∥∇θRp​\(φ2,P​\(θ1,ϕ1\)\)∥=∥∇θRp​\(φ2,P​\(θ1,ϕ1\)\)−∇θRp​\(φ2,P​\(θ2,ϕ2\)\)∥\\displaystyle\\leq\\lVert\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\)\\rVert=\\lVert\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{1\},\\phi\_\{1\}\)\)\-\\nabla\_\{\\theta\}R\_\{p\}\\big\(\\varphi\_\{2\},P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\rVert\(56\)≤Lθ​infπ∈Π​𝔼\(Z1,Z2\)∼π​∥Z1−Z2∥=Lθ​W​\(P​\(θ1,ϕ1\),P​\(θ2,ϕ2\)\)\.\\displaystyle\\leq L\_\{\\theta\}\\underset\{\\pi\\in\\Pi\}\{\\text\{inf\}\}\\ \\mathbb\{E\}\_\{\(Z\_\{1\},Z\_\{2\}\)\\sim\\pi\}\\lVert Z\_\{1\}\-Z\_\{2\}\\rVert\\ =L\_\{\\theta\}W\(P\(\\theta\_\{1\},\\phi\_\{1\}\),P\(\\theta\_\{2\},\\phi\_\{2\}\)\)\.Recall the definition ofφ1,φ2\\varphi\_\{1\},\\varphi\_\{2\}, so Eq\. \([53](https://arxiv.org/html/2605.29267#A6.E53)\) is proved\. Similarly, we can prove the same thing for modelqq, that is

∥argminϕ​Rq​\(ϕ,Q​\(θ1,ϕ1\)\)−argminϕ​Rq​\(ϕ,Q​\(θ2,ϕ2\)\)∥≤Lϕγϕ​W​\(Q​\(θ1,ϕ1\),Q​\(θ2,ϕ2\)\)\.\\lVert\\underset\{\\phi\}\{\\text\{argmin\}\}\\ R\_\{q\}\(\\phi,Q\(\\theta\_\{1\},\\phi\_\{1\}\)\)\-\\underset\{\\phi\}\{\\text\{argmin\}\}\\ R\_\{q\}\(\\phi,Q\(\\theta\_\{2\},\\phi\_\{2\}\)\)\\rVert\\leq\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\(\\theta\_\{1\},\\phi\_\{1\}\),Q\(\\theta\_\{2\},\\phi\_\{2\}\)\)\.\(57\)
Next, assume that the parameter\(θ,ϕ\)∈ℝn\(\\theta,\\phi\)\\in\\mathbb\{R\}^\{n\}and let𝒫​\(ℝn\)\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)be the probability distribution space consisting of all the distributions with finite first moment onℝn\\mathbb\{R\}^\{n\}\. All the mixture distributionsPPandQQsatisfyP,Q∈𝒫​\(ℝn\)P,Q\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)by lemma[F\.8](https://arxiv.org/html/2605.29267#A6.Thmtheorem8)\. It’s easy to check that\(𝒫​\(ℝn\),W\)\(\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\),W\)is a complete metric space with metricWW, the Wasserstein\(\-1\) distance\. For∀\(θ1,ϕ1\),\(θ2,ϕ2\)\\forall\\ \(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\), let

ri=\(\(θi,ϕi\),Pi,Qi\)∈ℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\),i∈\{1,2\}r\_\{i\}=\\big\(\(\\theta\_\{i\},\\phi\_\{i\}\),P\_\{i\},Q\_\{i\}\\big\)\\in\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\),i\\in\\\{1,2\\\}\(58\)be any two model parameter and data distribution pairs, andPi=P​\(θi−,ϕi−\)P\_\{i\}=P\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\),Qi=Q​\(θi−,ϕi−\)Q\_\{i\}=Q\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\)\.θi,ϕi\\theta\_\{i\},\\phi\_\{i\}are updated based on the previous round parametersθi−,ϕi−\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}and distributionsP​\(θi−,ϕi−\),Q​\(θi−,ϕi−\)P\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\),Q\(\\theta\_\{i\}^\{\-\},\\phi\_\{i\}^\{\-\}\)\. If we summarize the iterative process asri\+=Φ​\(ri\)r\_\{i\}^\{\+\}=\\Phi\(r\_\{i\}\)andri\+=\(\(θi\+,ϕi\+\),Pi\+,Qi\+\),Pi\+=P​\(θi,ϕi\),Qi\+=Q​\(θi,ϕi\)r\_\{i\}^\{\+\}=\(\(\\theta\_\{i\}^\{\+\},\\phi\_\{i\}^\{\+\}\),P\_\{i\}^\{\+\},Q\_\{i\}^\{\+\}\),P\_\{i\}^\{\+\}=P\(\\theta\_\{i\},\\phi\_\{i\}\),Q\_\{i\}^\{\+\}=Q\(\\theta\_\{i\},\\phi\_\{i\}\)fori∈\{1,2\}i\\in\\\{1,2\\\}, then we only need to prove thatΦ\\Phican be a compression mapping\. Let

v\\displaystyle v=\(∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥,W​\(P1,P2\),W​\(Q1,Q2\)\)T∈ℝ≥03,\\displaystyle=\(\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert,W\(P\_\{1\},P\_\{2\}\),W\(Q\_\{1\},Q\_\{2\}\)\)^\{T\}\\in\\mathbb\{R\}^\{3\}\_\{\\geq 0\},\(59\)v\+\\displaystyle v^\{\+\}=\(∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥,W​\(P1\+,P2\+\),W​\(Q1\+,Q2\+\)\)T∈ℝ≥03\.\\displaystyle=\(\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert,W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\),W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\)^\{T\}\\in\\mathbb\{R\}^\{3\}\_\{\\geq 0\}\.For convenience, we denote thatvectora≤ba\\leq bis equivalent to every element inaais less than or equal to that inbb\.We next prove that∃M∈ℝ3×3\\exists\\ M\\in\\mathbb\{R\}^\{3\\times 3\}such thatv\+≤M​vv^\{\+\}\\leq Mv\.

Considering three cases \(Eq\. \([21](https://arxiv.org/html/2605.29267#A6.E21)\)\-\([23](https://arxiv.org/html/2605.29267#A6.E23)\)\) in each iteration, there are total of nine cases for the two iteration paths\. For simplicity, denote

σθ=λℋθ\+λ𝒮θ=1−λℛθ,σϕ=λℋϕ\+λ𝒮ϕ=1−λℛϕ\\sigma\_\{\\theta\}=\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}=1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\ \\sigma\_\{\\phi\}=\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\phi\}=1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}as the ratios of synthetic data including synthetic curation data\. Table[2](https://arxiv.org/html/2605.29267#A6.T2)shows the nineMMmatrices corresponding to the nine cases\. Let’s start with the simplest case where\(θi\+,ϕi\+\)=G^3​\(θi,ϕi\)=\(Gp​\(θi,ϕi\),Gq​\(θi,ϕi\)\)\(\\theta\_\{i\}^\{\+\},\\phi\_\{i\}^\{\+\}\)=\\hat\{G\}\_\{3\}\(\\theta\_\{i\},\\phi\_\{i\}\)=\(G\_\{p\}\(\\theta\_\{i\},\\phi\_\{i\}\),G\_\{q\}\(\\theta\_\{i\},\\phi\_\{i\}\)\)\(Eq\. \([23](https://arxiv.org/html/2605.29267#A6.E23)\)\) fori∈\{1,2\}i\\in\\\{1,2\\\}\. Notice that

v\+=\[∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥W​\(P1\+,P2\+\)W​\(Q1\+,Q2\+\)\]≤\[Lθγθ​W​\(P1\+,P2\+\)\+Lϕγϕ​W​\(Q1\+,Q2\+\)W​\(P1\+,P2\+\)W​\(Q1\+,Q2\+\)\]b​y​E​q\.\([53](https://arxiv.org/html/2605.29267#A6.E53)\)​\([57](https://arxiv.org/html/2605.29267#A6.E57)\)\\displaystyle\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ v^\{\+\}=\\begin\{bmatrix\}\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\\\\ W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\\\\ W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\end\{bmatrix\}\\leq\\begin\{bmatrix\}\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\\\ W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\\\\ W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\end\{bmatrix\}\\ \\ \\ by\\ Eq\.~\(\\ref\{eq:apdix:fstPfEq4p\}\)\(\\ref\{eq:apdix:fstPfEq4q\}\)\(60\)≤\\displaystyle\\leq\[Lθγθ​W​\(P1\+,P2\+\)\+Lϕγϕ​W​\(Q1\+,Q2\+\)σθ​\[Kc​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\+\(1−λθϕ\)​\(Kc\+1\)​W​\(P1,P2\)\+λθϕ​\(Kc\+1\)​W​\(Q1,Q2\)\]σϕ​\[Kc​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\+λϕθ​\(Kc\+1\)​W​\(P1,P2\)\+\(1−λϕθ\)​\(Kc\+1\)​W​\(Q1,Q2\)\]\]b​y​L​e​m​m​a​\([F\.4](https://arxiv.org/html/2605.29267#A6.Thmtheorem4)\)\\displaystyle\\begin\{bmatrix\}\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\\\ \\sigma\_\{\\theta\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]\\\\ \\sigma\_\{\\phi\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]\\end\{bmatrix\}\\ \\ \\ by\\ Lemma~\(\\ref\{lem:disBound2Rnds\}\)≤\\displaystyle\\leqM3​v,M3=\(Lθγθ​σθ​Kc\+Lϕγϕ​σϕ​KcLθ​σθ​\(1−λθϕ\)​\(Kc\+1\)γθ\+Lϕ​σϕ​λϕθ​\(Kc\+1\)γϕLθ​σθ​λθϕ​\(Kc\+1\)γθ\+Lϕ​σϕ​\(1−λϕθ\)​\(Kc\+1\)γϕσθ​Kcσθ​\(1−λθϕ\)​\(Kc\+1\)σθ​λθϕ​\(Kc\+1\)σϕ​Kcσϕ​λϕθ​\(Kc\+1\)σϕ​\(1−λϕθ\)​\(Kc\+1\)\),\\displaystyle\\ M\_\{3\}v,\\ M\_\{3\}=\\left\(\\begin\{matrix\}\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\sigma\_\{\\theta\}K\_\{c\}\{\+\}\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\sigma\_\{\\phi\}K\_\{c\}&\\frac\{L\_\{\\theta\}\\sigma\_\{\\theta\}\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)\}\{\\gamma\_\{\\theta\}\}\{\+\}\\frac\{L\_\{\\phi\}\\sigma\_\{\\phi\}\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)\}\{\\gamma\_\{\\phi\}\}&\\frac\{L\_\{\\theta\}\\sigma\_\{\\theta\}\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)\}\{\\gamma\_\{\\theta\}\}\{\+\}\\frac\{L\_\{\\phi\}\\sigma\_\{\\phi\}\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)\}\{\\gamma\_\{\\phi\}\}\\\\ \\sigma\_\{\\theta\}K\_\{c\}&\\sigma\_\{\\theta\}\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)&\\sigma\_\{\\theta\}\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)\\\\ \\sigma\_\{\\phi\}K\_\{c\}&\\sigma\_\{\\phi\}\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)&\\sigma\_\{\\phi\}\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)\\end\{matrix\}\\right\),whereKc=L​K​\(1\+L​B2\)K\_\{c\}=LK\(1\+\\frac\{LB\}\{2\}\)as in Lemma[F\.4](https://arxiv.org/html/2605.29267#A6.Thmtheorem4)andLLis the Lipschitz constant for models and rewards,BBis the data space norm upper bound andKKis the curation sample number\.

Moreover, we prove the slightly more complex case, where both iterative paths update the same model first in this round\. Take\(θi\+,ϕi\+\)=G^1​\(θi,ϕi\)=\(Gp​\(θi,ϕi\),Gq​\(Gp​\(θi,ϕi\),ϕi\)\)\(\\theta\_\{i\}^\{\+\},\\phi\_\{i\}^\{\+\}\)=\\hat\{G\}\_\{1\}\(\\theta\_\{i\},\\phi\_\{i\}\)=\(G\_\{p\}\(\\theta\_\{i\},\\phi\_\{i\}\),G\_\{q\}\(G\_\{p\}\(\\theta\_\{i\},\\phi\_\{i\}\),\\phi\_\{i\}\)\)\(Eq\. \([21](https://arxiv.org/html/2605.29267#A6.E21)\)\) fori∈\{1,2\}i\\in\\\{1,2\\\}as the example\. Similar to Eq\. \([60](https://arxiv.org/html/2605.29267#A6.E60)\), we can get

v\+=\[∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥W​\(P1\+,P2\+\)W​\(Q1\+,Q2\+\)\]≤\[Lθγθ​W​\(P1\+,P2\+\)\+Lϕγϕ​W​\(Q​\(θ1\+,ϕ1\),Q​\(θ2\+,ϕ2\)\)W​\(P1\+,P2\+\)W​\(Q1\+,Q2\+\)\]\.\\displaystyle v^\{\+\}=\\begin\{bmatrix\}\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\\\\ W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\\\\ W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\end\{bmatrix\}\\leq\\begin\{bmatrix\}\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\),Q\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}\)\)\\\\ W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\\\\ W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\end\{bmatrix\}\.\(61\)The only difference between this vector and the vector in Eq\. \([60](https://arxiv.org/html/2605.29267#A6.E60)\) is the first element in the RHS of Eq\. \([61](https://arxiv.org/html/2605.29267#A6.E61)\)\. By the Corollary[F\.6](https://arxiv.org/html/2605.29267#A6.Thmtheorem6),

W​\(Q​\(θ1\+,ϕ1\),Q​\(θ2\+,ϕ2\)\)≤σϕ​\[Kc​∥\(θ1\+,ϕ1\)−\(θ2\+,ϕ2\)∥\+λϕθ​\(Kc\+1\)​W​\(P1,P2\)\+\(1−λϕθ\)​\(Kc\+1\)​W​\(Q1,Q2\)\]\\displaystyle W\(Q\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\),Q\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}\)\)\\leq\\sigma\_\{\\phi\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}\)\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]≤\\displaystyle\\leqσϕ​\[Kc​∥θ1\+−θ2\+∥\+Kc​∥ϕ1−ϕ2∥\+λϕθ​\(Kc\+1\)​W​\(P1,P2\)\+\(1−λϕθ\)​\(Kc\+1\)​W​\(Q1,Q2\)\]\\displaystyle\\ \\sigma\_\{\\phi\}\\big\[K\_\{c\}\\lVert\\theta\_\{1\}^\{\+\}\-\\theta\_\{2\}^\{\+\}\\rVert\+K\_\{c\}\\lVert\\phi\_\{1\}\-\\phi\_\{2\}\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]≤\\displaystyle\\leqσϕ​\[Kc​∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥\+Kc​∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\+λϕθ​\(Kc\+1\)​W​\(P1,P2\)\+\(1−λϕθ\)​\(Kc\+1\)​W​\(Q1,Q2\)\],\\displaystyle\\ \\sigma\_\{\\phi\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\+K\_\{c\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\],Therefore, substitute the inequality into the first row of Eq\. \([61](https://arxiv.org/html/2605.29267#A6.E61)\) and combine with Lemma[F\.4](https://arxiv.org/html/2605.29267#A6.Thmtheorem4), we have

∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥≤Lθγθ​W​\(P1\+,P2\+\)\+Lϕγϕ​W​\(Q​\(θ1\+,ϕ1\),Q​\(θ2\+,ϕ2\)\)\\displaystyle\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\\leq\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\),Q\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}\)\)\(62\)≤\\displaystyle\\leqLθγθσθ\[Kc∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\+\(1−λθϕ\)\(Kc\+1\)W\(P1,P2\)\+λθϕ\(Kc\+1\)W\(Q1,Q2\)\]\+Lϕγϕσϕ\[Kc\\displaystyle\\ \\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\sigma\_\{\\theta\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\sigma\_\{\\phi\}\\big\[K\_\{c\}∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥\+Kc∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\+λϕθ\(Kc\+1\)W\(P1,P2\)\+\(1−λϕθ\)\(Kc\+1\)W\(Q1,Q2\)\]\.\\displaystyle\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\+K\_\{c\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]\.Combine Eq\. \([62](https://arxiv.org/html/2605.29267#A6.E62)\) with Eq\. \([60](https://arxiv.org/html/2605.29267#A6.E60)\) and notice that ifLϕ​σϕ​Kcγϕ<1\\frac\{L\_\{\\phi\}\\sigma\_\{\\phi\}K\_\{c\}\}\{\\gamma\_\{\\phi\}\}<1, we can get

v\+≤M1​v,M1=\[γϕγϕ−σϕ​Lϕ​Kc​M3​\[1\]M3​\[2\]M3​\[3\]\],M3​\[i\]:i​th row of​M3,i∈\{1,2,3\}\.v^\{\+\}\\leq M\_\{1\}v,\\ M\_\{1\}=\\begin\{bmatrix\}\\frac\{\\gamma\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-\\sigma\_\{\\phi\}L\_\{\\phi\}K\_\{c\}\}M\_\{3\}\[1\]\\\\ M\_\{3\}\[2\]\\\\ M\_\{3\}\[3\]\\end\{bmatrix\},\\ M\_\{3\}\[i\]:i\\ \\text\{th row of \}M\_\{3\},\\ i\\in\\\{1,2,3\\\}\.\(63\)Similarly, if\(θi\+,ϕi\+\)=G^2​\(θi,ϕi\)=\(Gp​\(θi,Gq​\(θi,ϕi\)\),Gq​\(θi,ϕi\)\)\(\\theta\_\{i\}^\{\+\},\\phi\_\{i\}^\{\+\}\)=\\hat\{G\}\_\{2\}\(\\theta\_\{i\},\\phi\_\{i\}\)=\(G\_\{p\}\(\\theta\_\{i\},G\_\{q\}\(\\theta\_\{i\},\\phi\_\{i\}\)\),G\_\{q\}\(\\theta\_\{i\},\\phi\_\{i\}\)\), we can get theM2M\_\{2\}matrix thatv\+≤M2​vv^\{\+\}\\leq M\_\{2\}vin Table[2](https://arxiv.org/html/2605.29267#A6.T2)\.

Table 2:Compression matrices for 9 possible iteration cases\.M3​\[i\]:i​th row of​M3,i∈\{1,2,3\}M\_\{3\}\[i\]:i\\ \\text\{th row of \}M\_\{3\},\\ i\\in\\\{1,2,3\\\}\.Now we show the proof of the most complex cases, where two iterative paths\(θ1,ϕ1\),\(θ2,ϕ2\)\(\\theta\_\{1\},\\phi\_\{1\}\),\(\\theta\_\{2\},\\phi\_\{2\}\)have different update order in this round\. Take\(θ1\+,ϕ1\+\)=G^1​\(θ1,ϕ1\)=\(Gp​\(θ1,ϕ1\),Gq​\(Gp​\(θ1,ϕ1\),ϕ1\)\)\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)=\\hat\{G\}\_\{1\}\(\\theta\_\{1\},\\phi\_\{1\}\)=\(G\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),G\_\{q\}\(G\_\{p\}\(\\theta\_\{1\},\\phi\_\{1\}\),\\phi\_\{1\}\)\)\(Eq\. \([21](https://arxiv.org/html/2605.29267#A6.E21)\)\) and\(θ2\+,ϕ2\+\)=G^2​\(θ2,ϕ2\)=\(Gp​\(θ2,Gq​\(θ2,ϕ2\)\),Gq​\(θ2,ϕ2\)\)\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)=\\hat\{G\}\_\{2\}\(\\theta\_\{2\},\\phi\_\{2\}\)=\(G\_\{p\}\(\\theta\_\{2\},G\_\{q\}\(\\theta\_\{2\},\\phi\_\{2\}\)\),G\_\{q\}\(\\theta\_\{2\},\\phi\_\{2\}\)\)\(Eq\. \([22](https://arxiv.org/html/2605.29267#A6.E22)\)\) as the example\. Similarly, in this case we can get

v\+=\[∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥W​\(P1\+,P2\+\)W​\(Q1\+,Q2\+\)\]≤\[Lθγθ​W​\(P1\+,P​\(θ2,ϕ2\+\)\)\+Lϕγϕ​W​\(Q​\(θ1\+,ϕ1\),Q2\+\)W​\(P1\+,P2\+\)W​\(Q1\+,Q2\+\)\]\.\\displaystyle v^\{\+\}=\\begin\{bmatrix\}\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\\\\ W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\\\\ W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\end\{bmatrix\}\\leq\\begin\{bmatrix\}\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\_\{1\}^\{\+\},P\(\\theta\_\{2\},\\phi\_\{2\}^\{\+\}\)\)\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\),Q\_\{2\}^\{\+\}\)\\\\ W\(P\_\{1\}^\{\+\},P\_\{2\}^\{\+\}\)\\\\ W\(Q\_\{1\}^\{\+\},Q\_\{2\}^\{\+\}\)\\end\{bmatrix\}\.\(64\)Only the first row is different from the previous case proof, and by Corollary[F\.6](https://arxiv.org/html/2605.29267#A6.Thmtheorem6)we have

∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥≤Lθγθ​W​\(P1\+,P​\(θ2,ϕ2\+\)\)\+Lϕγϕ​W​\(Q​\(θ1\+,ϕ1\),Q2\+\)\\displaystyle\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\\leq\\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}W\(P\_\{1\}^\{\+\},P\(\\theta\_\{2\},\\phi\_\{2\}^\{\+\}\)\)\+\\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}W\(Q\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\),Q\_\{2\}^\{\+\}\)\(65\)≤\\displaystyle\\leqLθγθ​σθ​\[Kc​∥\(θ1,ϕ1\)−\(θ2,ϕ2\+\)∥\+\(1−λθϕ\)​\(Kc\+1\)​W​\(P1,P2\)\+λθϕ​\(Kc\+1\)​W​\(Q1,Q2\)\]\\displaystyle\\ \\frac\{L\_\{\\theta\}\}\{\\gamma\_\{\\theta\}\}\\sigma\_\{\\theta\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}^\{\+\}\)\\rVert\+\(1\-\\lambda\_\{\\theta\}^\{\\phi\}\)\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\\lambda\_\{\\theta\}^\{\\phi\}\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]\+\\displaystyle\+Lϕγϕ​σϕ​\[Kc​∥\(θ1\+,ϕ1\)−\(θ2,ϕ2\)∥\+λϕθ​\(Kc\+1\)​W​\(P1,P2\)\+\(1−λϕθ\)​\(Kc\+1\)​W​\(Q1,Q2\)\]\\displaystyle\\ \\frac\{L\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\}\\sigma\_\{\\phi\}\\big\[K\_\{c\}\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+\\lambda\_\{\\phi\}^\{\\theta\}\(K\_\{c\}\+1\)W\(P\_\{1\},P\_\{2\}\)\+\(1\-\\lambda\_\{\\phi\}^\{\\theta\}\)\(K\_\{c\}\+1\)W\(Q\_\{1\},Q\_\{2\}\)\\big\]≤\\displaystyle\\leq\(σθ​Lθ​Kcγθ\+σϕ​Lϕ​Kcγϕ\)​∥\(θ1\+,ϕ1\+\)−\(θ2\+,ϕ2\+\)∥\+M3​\[1\]​v,\\displaystyle\\ \(\\frac\{\\sigma\_\{\\theta\}L\_\{\\theta\}K\_\{c\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{\\sigma\_\{\\phi\}L\_\{\\phi\}K\_\{c\}\}\{\\gamma\_\{\\phi\}\}\)\\lVert\(\\theta\_\{1\}^\{\+\},\\phi\_\{1\}^\{\+\}\)\-\(\\theta\_\{2\}^\{\+\},\\phi\_\{2\}^\{\+\}\)\\rVert\+M\_\{3\}\[1\]v,therefore, ifσθ​Lθ​Kcγθ\+σϕ​Lϕ​Kcγϕ<1\\frac\{\\sigma\_\{\\theta\}L\_\{\\theta\}K\_\{c\}\}\{\\gamma\_\{\\theta\}\}\+\\frac\{\\sigma\_\{\\phi\}L\_\{\\phi\}K\_\{c\}\}\{\\gamma\_\{\\phi\}\}<1, then

v\+≤M12​v,M12=\[γθ​γϕγθ​γϕ−γϕ​σθ​Lθ​Kc−γθ​σϕ​Lϕ​Kc×M3​\[1\]M3​\[2\]M3​\[3\]\],M3​\[i\]:i​th row of​M3,i∈\{1,2,3\}\.v^\{\+\}\\leq M\_\{12\}v,\\ M\_\{12\}=\\begin\{bmatrix\}\\frac\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\-\\gamma\_\{\\phi\}\\sigma\_\{\\theta\}L\_\{\\theta\}K\_\{c\}\-\\gamma\_\{\\theta\}\\sigma\_\{\\phi\}L\_\{\\phi\}K\_\{c\}\}\\times M\_\{3\}\[1\]\\\\ M\_\{3\}\[2\]\\\\ M\_\{3\}\[3\]\\end\{bmatrix\},\\ M\_\{3\}\[i\]:i\\ \\text\{th row of \}M\_\{3\},\\ i\\in\\\{1,2,3\\\}\.\(66\)Similarly, we can estimate the compression matrices of rest cases and the results are shown in Table[2](https://arxiv.org/html/2605.29267#A6.T2)\.

Notice that for compression matrixM3M\_\{3\}in Eq\. \([60](https://arxiv.org/html/2605.29267#A6.E60)\), ifσθ,σϕ→0\\sigma\_\{\\theta\},\\sigma\_\{\\phi\}\\rightarrow 0which means improving the real data ratiosλℋθ,λℋϕ\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\phi\}, then every element inM3M\_\{3\}converges to 0\. Since all elements inM3M\_\{3\}are positive and are linear combinations ofσθ,σϕ\\sigma\_\{\\theta\},\\sigma\_\{\\phi\}and constants, let

τ1=γθ​γϕ\(Kc\+1\)​\(Lθ​γϕ\+Lϕ​γθ\+2​γθ​γϕ\)∈\(0,1\),\\tau\_\{1\}=\\frac\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{\(K\_\{c\}\+1\)\(L\_\{\\theta\}\\gamma\_\{\\phi\}\+L\_\{\\phi\}\\gamma\_\{\\theta\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\)\}\\in\(0,1\),\(67\)and it’s easy to check that ifσθ,σϕ<τ1\\sigma\_\{\\theta\},\\sigma\_\{\\phi\}<\\tau\_\{1\}, then

∥M3∥1=max1≤j≤3​∑i=13\|M3​\[i,j\]\|<1,\\lVert M\_\{3\}\\rVert\_\{1\}=\\underset\{1\\leq j\\leq 3\}\{\\text\{max\}\}\\ \\sum\_\{i=1\}^\{3\}\|M\_\{3\}\[i,j\]\|<1,where∥∥1\\lVert\\rVert\_\{1\}is the 1\-norm\. For other compression matrices, from their expressions in Table[2](https://arxiv.org/html/2605.29267#A6.T2), we can see that the coefficients beforeM3​\[1\]M\_\{3\}\[1\]all monotonically converge to 1 asσθ,σϕ→0\\sigma\_\{\\theta\},\\sigma\_\{\\phi\}\\rightarrow 0\. Therefore,∃τ2\>0\\exists\\ \\tau\_\{2\}\>0such that ifσθ,σϕ<τ2\\sigma\_\{\\theta\},\\sigma\_\{\\phi\}<\\tau\_\{2\}, then

maxk≠3​∥Mk∥1<1\.\\underset\{k\\neq 3\}\{\\text\{max\}\}\\ \\lVert M\_\{k\}\\rVert\_\{1\}<1\.It’s easy to check that

τ2=min\(\(γθ​Lϕ\+γϕ​Lθ\)​2​Kc\+1Kc\+1\+2​γθ​γϕ4​Kc​\(γθ​Lϕ\+γϕ​Lθ\),γθ​γϕKc​\(γθ​Lϕ\+γϕ​Lθ\),\\displaystyle\\tau\_\{2\}=\\min\\Big\(\\frac\{\(\\gamma\_\{\\theta\}L\_\{\\phi\}\+\\gamma\_\{\\phi\}L\_\{\\theta\}\)\\frac\{2K\_\{c\}\+1\}\{K\_\{c\}\+1\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{4K\_\{c\}\(\\gamma\_\{\\theta\}L\_\{\\phi\}\+\\gamma\_\{\\phi\}L\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{K\_\{c\}\(\\gamma\_\{\\theta\}L\_\{\\phi\}\+\\gamma\_\{\\phi\}L\_\{\\theta\}\)\},γϕ​Lθ​2​Kc\+1Kc\+1\+2​γθ​γϕ4​Kc​γϕ​Lθ,\\displaystyle\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\frac\{2K\_\{c\}\+1\}\{K\_\{c\}\+1\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{4K\_\{c\}\\gamma\_\{\\phi\}L\_\{\\theta\}\},\(68\)γθKc​Lθ,γθ​Lϕ​2​Kc\+1Kc\+1\+2​γθ​γϕ4​Kc​γθ​Lϕ,γϕKc​Lϕ\)\\displaystyle\\frac\{\\gamma\_\{\\theta\}\}\{K\_\{c\}L\_\{\\theta\}\},\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\frac\{2K\_\{c\}\+1\}\{K\_\{c\}\+1\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{4K\_\{c\}\\gamma\_\{\\theta\}L\_\{\\phi\}\},\\frac\{\\gamma\_\{\\phi\}\}\{K\_\{c\}L\_\{\\phi\}\}\\Big\)satisfies this condition\. Letτ=min⁡\(τ1,τ2\)∈\(0,1\)\\tau=\\min\(\\tau\_\{1\},\\tau\_\{2\}\)\\in\(0,1\)in Eq\. \([67](https://arxiv.org/html/2605.29267#A6.E67)\) and \([68](https://arxiv.org/html/2605.29267#A6.E68)\), then for any cases we have

∥v\+∥1≤max𝑘​∥Mk∥1​∥v∥1<∥v∥1\.\\lVert v^\{\+\}\\rVert\_\{1\}\\leq\\underset\{k\}\{\\text\{max\}\}\\ \\lVert M\_\{k\}\\rVert\_\{1\}\\lVert v\\rVert\_\{1\}<\\lVert v\\rVert\_\{1\}\.\(69\)Defined∗​\(r1,r2\)=∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥\+W​\(P1,P2\)\+W​\(Q1,Q2\)d^\{\*\}\(r\_\{1\},r\_\{2\}\)=\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert\+W\(P\_\{1\},P\_\{2\}\)\+W\(Q\_\{1\},Q\_\{2\}\)for anyr1=\(\(θ1,ϕ1\),P1,Q1\)r\_\{1\}=\(\(\\theta\_\{1\},\\phi\_\{1\}\),P\_\{1\},Q\_\{1\}\),r2=\(\(θ2,ϕ2\),P2,Q2\)∈ℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\)r\_\{2\}=\(\(\\theta\_\{2\},\\phi\_\{2\}\),P\_\{2\},Q\_\{2\}\)\\in\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\. It’s easy to check this is a metric defined onℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\)\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\. By Eq\. \([69](https://arxiv.org/html/2605.29267#A6.E69)\) and Eq\. \([58](https://arxiv.org/html/2605.29267#A6.E58)\), notice that

∥v\+∥1=d∗​\(Φ​\(r1\),Φ​\(r2\)\)=d∗​\(r1\+,r2\+\)≤max𝑘​∥Mk∥1​∥v∥1=max𝑘​∥Mk∥1​d∗​\(r1,r2\)<d∗​\(r1,r2\)\.\\lVert v^\{\+\}\\rVert\_\{1\}=d^\{\*\}\(\\Phi\(r\_\{1\}\),\\Phi\(r\_\{2\}\)\)=d^\{\*\}\(r\_\{1\}^\{\+\},r\_\{2\}^\{\+\}\)\\leq\\underset\{k\}\{\\text\{max\}\}\\ \\lVert M\_\{k\}\\rVert\_\{1\}\\lVert v\\rVert\_\{1\}=\\underset\{k\}\{\\text\{max\}\}\\ \\lVert M\_\{k\}\\rVert\_\{1\}d^\{\*\}\(r\_\{1\},r\_\{2\}\)<d^\{\*\}\(r\_\{1\},r\_\{2\}\)\.We only need to prove that\(ℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\),d∗\)\\big\(\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\),d^\{\*\}\\big\)is a complete metric space, then by Banach fixed point theorem, we can prove our conclusion\.

Let\{rn\}n=1∞=\{\(θn,ϕn\),Pn,Qn\)\}\\\{r\_\{n\}\\\}\_\{n=1\}^\{\\infty\}=\\\{\(\\theta\_\{n\},\\phi\_\{n\}\),P\_\{n\},Q\_\{n\}\)\\\}be any Cauchy sequence in\(ℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\),d∗\)\\big\(\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\),d^\{\*\}\\big\), that is

∀ε\>0,∃N∈ℤ\+,s\.t\.∀n,m≥N,d∗​\(rn,rm\)<ε\.\\forall\\ \\varepsilon\>0,\\ \\exists\\ N\\in\\mathbb\{Z\}^\{\+\},\\ s\.t\.\\ \\forall\\ n,m\\geq N,d^\{\*\}\(r\_\{n\},r\_\{m\}\)<\\varepsilon\.Thus, for anyn,m≥Nn,m\\geq N,

∥\(θ1,ϕ1\)−\(θ2,ϕ2\)∥<ε,W​\(P1,P2\)<ε,W​\(Q1,Q2\)<ε\\lVert\(\\theta\_\{1\},\\phi\_\{1\}\)\-\(\\theta\_\{2\},\\phi\_\{2\}\)\\rVert<\\varepsilon,\\ W\(P\_\{1\},P\_\{2\}\)<\\varepsilon,\\ W\(Q\_\{1\},Q\_\{2\}\)<\\varepsilonwhich means\{\(θn,ϕn\)\}\\\{\(\\theta\_\{n\},\\phi\_\{n\}\)\\\},\{Pn\}\\\{P\_\{n\}\\\}and\{Qn\}\\\{Q\_\{n\}\\\}are all Cauchy sequences in their spaces\. Since\(ℝn,∥∥\)\(\\mathbb\{R\}^\{n\},\\lVert\\rVert\)and\(𝒫​\(ℝn\),W\)\(\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\),W\)are complete metric spaces, there exists\(θ∗,ϕ∗\)∈ℝn\(\\theta\_\{\*\},\\phi\_\{\*\}\)\\in\\mathbb\{R\}^\{n\},P∗∈𝒫​\(ℝn\)P\_\{\*\}\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)andQ∗∈𝒫​\(ℝn\)Q\_\{\*\}\\in\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)such that\(θn,ϕn\)→\(θ∗,ϕ∗\)\(\\theta\_\{n\},\\phi\_\{n\}\)\\rightarrow\(\\theta\_\{\*\},\\phi\_\{\*\}\),W​\(Pn,P∗\)→0W\(P\_\{n\},P\_\{\*\}\)\\rightarrow 0andW​\(Qn,Q∗\)→0W\(Q\_\{n\},Q\_\{\*\}\)\\rightarrow 0\. Definer∗=\(\(θ∗,ϕ∗\),P∗,Q∗\)∈ℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\)r\_\{\*\}=\(\(\\theta\_\{\*\},\\phi\_\{\*\}\),P\_\{\*\},Q\_\{\*\}\)\\in\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\), thend∗​\(rn,r∗\)→0d^\{\*\}\(r\_\{n\},r\_\{\*\}\)\\rightarrow 0\. Therefore,\(ℝn×𝒫​\(ℝn\)×𝒫​\(ℝn\),d∗\)\\big\(\\mathbb\{R\}^\{n\}\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\)\\times\\mathcal\{P\}\(\\mathbb\{R\}^\{n\}\),d^\{\*\}\\big\)is a complete metric space\.

In summary, if the real data ratios greater than1−τ∈\(0,1\)1\-\\tau\\in\(0,1\), then the iterating stable point\(θ∗,ϕ∗\)\(\\theta^\{\*\},\\phi^\{\*\}\)always exists and mixture distribution converges to stable distributions\. By the definitions of the mixture distributionsPPandQQ, it’s easy to check thatPtx,y→P​\(θ∗,ϕ∗\)P\_\{t\}^\{x,y\}\\rightarrow P\(\\theta^\{\*\},\\phi^\{\*\}\)andQtx,y→Q​\(θ∗,ϕ∗\)Q\_\{t\}^\{x,y\}\\rightarrow Q\(\\theta^\{\*\},\\phi^\{\*\}\)ast→∞t\\rightarrow\\infty\. By Eq\. \([67](https://arxiv.org/html/2605.29267#A6.E67)\) and \([68](https://arxiv.org/html/2605.29267#A6.E68)\),τ∈\(0,1\)\\tau\\in\(0,1\)is a constantrelated toKKvalue in the curation process, network architectures, datasets, loss functions’ choice\. Formally,

τ=min⁡\(τ1,τ2\)\\displaystyle\\tau=\\min\(\\tau\_\{1\},\\tau\_\{2\}\)=min\(γθ​γϕ\(Kc\+1\)​\(Lθ​γϕ\+Lϕ​γθ\+2​γθ​γϕ\),γϕKc​Lϕ,γθKc​Lθ,γθ​γϕKc​\(γθ​Lϕ\+γϕ​Lθ\),\\displaystyle=\\min\\Big\(\\frac\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{\(K\_\{c\}\+1\)\(L\_\{\\theta\}\\gamma\_\{\\phi\}\+L\_\{\\phi\}\\gamma\_\{\\theta\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\)\},\\frac\{\\gamma\_\{\\phi\}\}\{K\_\{c\}L\_\{\\phi\}\},\\frac\{\\gamma\_\{\\theta\}\}\{K\_\{c\}L\_\{\\theta\}\},\\frac\{\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{K\_\{c\}\(\\gamma\_\{\\theta\}L\_\{\\phi\}\+\\gamma\_\{\\phi\}L\_\{\\theta\}\)\},\(γθ​Lϕ\+γϕ​Lθ\)​2​Kc\+1Kc\+1\+2​γθ​γϕ4​Kc​\(γθ​Lϕ\+γϕ​Lθ\),γϕ​Lθ​2​Kc\+1Kc\+1\+2​γθ​γϕ4​Kc​γϕ​Lθ,γθ​Lϕ​2​Kc\+1Kc\+1\+2​γθ​γϕ4​Kc​γθ​Lϕ\)\.\\displaystyle\\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\ \\frac\{\(\\gamma\_\{\\theta\}L\_\{\\phi\}\+\\gamma\_\{\\phi\}L\_\{\\theta\}\)\\frac\{2K\_\{c\}\+1\}\{K\_\{c\}\+1\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{4K\_\{c\}\(\\gamma\_\{\\theta\}L\_\{\\phi\}\+\\gamma\_\{\\phi\}L\_\{\\theta\}\)\},\\frac\{\\gamma\_\{\\phi\}L\_\{\\theta\}\\frac\{2K\_\{c\}\+1\}\{K\_\{c\}\+1\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{4K\_\{c\}\\gamma\_\{\\phi\}L\_\{\\theta\}\},\\frac\{\\gamma\_\{\\theta\}L\_\{\\phi\}\\frac\{2K\_\{c\}\+1\}\{K\_\{c\}\+1\}\+2\\gamma\_\{\\theta\}\\gamma\_\{\\phi\}\}\{4K\_\{c\}\\gamma\_\{\\theta\}L\_\{\\phi\}\}\\Big\)\.∎

### F\.9Proof of Theorem[4\.8](https://arxiv.org/html/2605.29267#S4.Thmtheorem8)

Theorem[4\.8](https://arxiv.org/html/2605.29267#S4.Thmtheorem8)\.Under the conditions of Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4)and Assumption[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5), for anyλ^ℛθ,λ^ℛϕ\>τ\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\},\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\phi\}\>\\tauas in Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4), we have

\|Jp​\(θ∗​\(λ^ℛθ\)\)−Jp​\(θ∗​\(1\)\)\|=𝒪​\(1−λ^ℛθ\),\\displaystyle\\left\|J\_\{p\}\\left\(\\theta^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\right\)\\right\)\-J\_\{p\}\\left\(\\theta^\{\*\}\\Big\(1\\Big\)\\right\)\\right\|=\\mathcal\{O\}\\Big\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\Big\),\|Jq​\(ϕ∗​\(λ^ℛϕ\)\)−Jq​\(ϕ∗​\(1\)\)\|=𝒪​\(1−λ^ℛϕ\)\.\\displaystyle\\left\|J\_\{q\}\\left\(\\phi^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\phi\}\\right\)\\right\)\-J\_\{q\}\\Big\(\\phi^\{\*\}\\Big\(1\\Big\)\\Big\)\\right\|=\\mathcal\{O\}\\Big\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\phi\}\\Big\)\.
###### Proof\.

First, we prove that if the real data ratiosλℛθ,λℛϕ\>τ\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\>\\tauas in Proposition[3\.4](https://arxiv.org/html/2605.29267#S3.Thmtheorem4), thenJp​\(θ\),Jq​\(ϕ\)J\_\{p\}\(\\theta\),J\_\{q\}\(\\phi\)are Lipschitz inθ,ϕ\\theta,\\phi, respectively\. Sinceλℛθ,λℛϕ\>τ\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{R\}\}^\{\\phi\}\>\\tau, the stable pointsθ∗,ϕ∗\\theta^\{\*\},\\phi^\{\*\}exist\. For anyθ1,θ2∈Θ\\theta\_\{1\},\\theta\_\{2\}\\in\\Theta, letyybe a random variable that follows𝒟θ\\mathcal\{D\}\_\{\\theta\}distribution andx1=pθ1​\(x\|y\),x2=pθ2​\(x\|y\)x\_\{1\}=p\_\{\\theta\_\{1\}\}\(x\|y\),x\_\{2\}=p\_\{\\theta\_\{2\}\}\(x\|y\)and byJpJ\_\{p\}’s definition in Eq\.\([8](https://arxiv.org/html/2605.29267#S4.E8)\) and rewards’ Lipschitz property, we can get

\|Jp​\(θ1\)−Jp​\(θ2\)\|\\displaystyle\|J\_\{p\}\(\\theta\_\{1\}\)\-J\_\{p\}\(\\theta\_\{2\}\)\|=\|𝔼​\[rθ​\(x1,y\)−rθ​\(x2,y\)\]\|≤L​𝔼​\[\|x1−x2\|\]=L​𝔼y∼𝒟θ​\[\|pθ1​\(y\)−pθ2​\(y\)\|\]≤L2​∥θ1−θ2∥,\\displaystyle=\|\\mathbb\{E\}\[r\_\{\\theta\}\(x\_\{1\},y\)\-r\_\{\\theta\}\(x\_\{2\},y\)\]\|\\leq L\\mathbb\{E\}\[\|x\_\{1\}\-x\_\{2\}\|\]=L\\mathbb\{E\}\_\{y\\sim\\mathcal\{D\}\_\{\\theta\}\}\[\|p\_\{\\theta\_\{1\}\}\(y\)\-p\_\{\\theta\_\{2\}\}\(y\)\|\]\\leq L^\{2\}\\lVert\\theta\_\{1\}\-\\theta\_\{2\}\\rVert,whereLLis the Lipschitz parameter for models\. Therefore,JpJ\_\{p\}is Lipschitz inθ\\theta, and∥∇θJp​\(θ∗\)∥≤supθ∈Θ​∥∇θJp∥≤L2\\lVert\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)\\rVert\\leq\\underset\{\\theta\\in\\Theta\}\{\\sup\}\\lVert\\nabla\_\{\\theta\}J\_\{p\}\\rVert\\leq L^\{2\}\. Next, similar to the proof of Proposition[4\.1](https://arxiv.org/html/2605.29267#S4.Thmtheorem1), we have

∂θ∗∂λℛθ\\displaystyle\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}=−Sp​\(∂Fp∂λℛθ\+Cp​∂Fq∂λℛθ\),∂ϕ∗∂λℛθ=−Sq​\(Cq​∂Fp∂λℛθ\+∂Fq∂λℛθ\)\.\\displaystyle=\-S\_\{p\}\\left\(\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\+C\_\{p\}\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\\right\),\\ \\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}=\-S\_\{q\}\\left\(C\_\{q\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\\right\)\.\(70\)∂θ∗∂λ𝒮θ\\displaystyle\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}=−Sp​\(∂Fp∂λ𝒮θ\+Cp​∂Fq∂λ𝒮θ\),∂ϕ∗∂λ𝒮θ=−Sq​\(Cq​∂Fp∂λ𝒮θ\+∂Fq∂λ𝒮θ\),\\displaystyle=\-S\_\{p\}\\left\(\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\+C\_\{p\}\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\\right\),\\ \\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}=\-S\_\{q\}\\left\(C\_\{q\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\\right\),∂θ∗∂λℋθ\\displaystyle\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=−Sp​\(∂Fp∂λℋθ\+Cp​∂Fq∂λℋθ\),∂ϕ∗∂λℋθ=−Sq​\(Cq​∂Fp∂λℋθ\+∂Fq∂λℋθ\)\.\\displaystyle=\-S\_\{p\}\\left\(\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+C\_\{p\}\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\),\\ \\frac\{\\partial\\phi^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=\-S\_\{q\}\\left\(C\_\{q\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\+\\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\right\)\.Moreover, similar to the proof in Theorem[4\.5](https://arxiv.org/html/2605.29267#S4.Thmtheorem5)and the extension proof in Theorem[E\.3](https://arxiv.org/html/2605.29267#A5.Thmtheorem3)\(Eq\. \([F\.4](https://arxiv.org/html/2605.29267#A6.Ex75)\),\([E\.3](https://arxiv.org/html/2605.29267#A5.Ex39)\)\), ifλℛθ\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}is changed by increasing or decreasing the sample size of real data, it’s easy to check thatd​λ𝒮θ=−λ𝒮θλℋθ\+λ𝒮θ​d​λℛθd\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}=\-\\frac\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},d​λℋθ=−λℋθλℋθ\+λ𝒮θ​d​λℛθd\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}=\-\\frac\{\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\+\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}d\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}, and

∂Fp∂λℛθ=11−λℛθ​𝔼ℛθ\\displaystyle\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}=\\frac\{1\}\{1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[∇θℓθ​\(θ∗\)\]=𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]−𝔼P​\(θ∗,ϕ∗\)/ℛθ​\[∇θℓθ​\(θ∗\)\],∂Fq∂λℛθ=0,\\displaystyle\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]=\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\-\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\],\\ \\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}=0,\(71\)∂Fp∂λ𝒮θ\\displaystyle\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}=−1λ𝒮θ​𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]=−1−λℛθλ𝒮θ​∂Fp∂λℛθ,∂Fq∂λ𝒮θ=0,\\displaystyle=\-\\frac\{1\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]=\-\\frac\{1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\},\\ \\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}=0,∂Fp∂λℋθ\\displaystyle\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=−1λℋθ​𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]=−1−λℛθλℋθ​∂Fp∂λℛθ,∂Fq∂λℋθ=0,\\displaystyle=\-\\frac\{1\}\{\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]=\-\\frac\{1\-\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\{\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\},\\ \\frac\{\\partial F\_\{q\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}=0,whereP​\(θ∗,ϕ∗\)/ℛθP\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}represents the distribution of synthetic and curated synthetic data with model parametersθ∗\\theta^\{\*\}andϕ∗\\phi^\{\*\}\. Notice that ifλℛθ\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}is changed by other reasons, for example reducing the size of non\-real datasetsℋθ\\mathcal\{H\}^\{\\theta\}and𝒮θ\\mathcal\{S\}^\{\\theta\}, we can prove this theorem similar to the methods in Appendix[E\.3](https://arxiv.org/html/2605.29267#A5.SS3)\.

For anyλ^ℛθ\>τ\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\>\\taufixed, denote the other two mixing weights asλ^𝒮θ,λ^ℋθ\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\},\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}andλ^ℛθ\+λ^𝒮θ\+λ^ℋθ=1\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\+\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\}\+\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}=1\. For simplicity, since different\(λℛθ,λ𝒮θ,λℋθ\)\(\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\},\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\)decides differentθ∗\\theta^\{\*\}, we define

𝒥​\(t\)=Jp​\(θ∗​\(1\+t​\(λ^ℛθ−1\),t​λ^𝒮θ,t​λ^ℋθ\)\):\[0,1\]→ℝ,𝒥​\(1\)=Jp​\(θ∗​\(λ^ℛθ\)\),𝒥​\(0\)=Jp​\(θ∗​\(1\)\)\.\\mathcal\{J\}\(t\)=J\_\{p\}\\left\(\\theta^\{\*\}\\left\(1\+t\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1\),t\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\},t\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}\\right\)\\right\):\[0,1\]\\rightarrow\\mathbb\{R\},\\ \\mathcal\{J\}\(1\)=J\_\{p\}\\left\(\\theta^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\right\)\\right\),\\ \\mathcal\{J\}\(0\)=J\_\{p\}\\left\(\\theta^\{\*\}\\left\(1\\right\)\\right\)\.We ignore the datasetDθD\_\{\\theta\}in this notation since it’s fixed\.𝒥​\(t\)\\mathcal\{J\}\(t\)is differentiable with respect tottguaranteed byJpJ\_\{p\}and its definition\. By combining Eq\. \([70](https://arxiv.org/html/2605.29267#A6.E70)\) and Eq\. \([71](https://arxiv.org/html/2605.29267#A6.E71)\), we have that for anyλ^ℛθ\>τ\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\>\\tauregardless of the values ofλ^𝒮θ,λ^ℋθ\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\},\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\},

\|Jp​\(θ∗​\(λ^ℛθ\)\)−Jp​\(θ∗​\(1\)\)\|=\|𝒥​\(1\)−𝒥​\(0\)\|=\|∫t=01dd​t​𝒥​\(t\)​𝑑t\|\\displaystyle\\left\|J\_\{p\}\\left\(\\theta^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\right\)\\right\)\-J\_\{p\}\\left\(\\theta^\{\*\}\\Big\(1\\Big\)\\right\)\\right\|=\\left\|\\mathcal\{J\}\(1\)\-\\mathcal\{J\}\(0\)\\right\|=\\left\|\\int\_\{t=0\}^\{1\}\\frac\{d\}\{dt\}\\mathcal\{J\}\(t\)dt\\right\|=\\displaystyle=\\\|∫t=01⟨∇θJp​\(θ∗\),dd​t​θ∗​\(1\+t​\(λ^ℛθ−1\),t​λ^𝒮θ,t​λ^ℋθ\)⟩​𝑑t\|\\displaystyle\\left\|\\int\_\{t=0\}^\{1\}\\left\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\frac\{d\}\{dt\}\\theta^\{\*\}\\left\(1\+t\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1\),t\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\},t\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}\\right\)\\right\\rangle dt\\right\|=\\displaystyle=\\\|∫t=01⟨∇θJp​\(θ∗\),\(∂θ∗∂λℛθ​\(1\+t​\(λ^ℛθ−1\)\),∂θ∗∂λ𝒮θ​\(t​λ^𝒮θ\),∂θ∗∂λℋθ​\(t​λ^ℋθ\)\)​\(λ^ℛθ−1,λ^𝒮θ,λ^ℋθ\)T⟩​𝑑t\|\\displaystyle\\left\|\\int\_\{t=0\}^\{1\}\\left\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\left\(\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\\left\(1\+t\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1\)\\right\),\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{S\}\}^\{\\theta\}\}\\left\(t\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\}\\right\),\\frac\{\\partial\\theta^\{\*\}\}\{\\partial\\lambda\_\{\\mathcal\{H\}\}^\{\\theta\}\}\\left\(t\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}\\right\)\\right\)\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1,\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\},\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}\\right\)^\{T\}\\right\\rangle dt\\right\|=\\displaystyle=\\\|∫t=01⟨∇θJp​\(θ∗\),\(−Sp​ℱ,t​\(λ^ℛθ−1\)t​λ^𝒮θ​Sp​ℱ,t​\(λ^ℛθ−1\)t​λ^ℋθ​Sp​ℱ\)​\(λ^ℛθ−1,λ^𝒮θ,λ^ℋθ\)T⟩​𝑑t\|by Eq\. \([70](https://arxiv.org/html/2605.29267#A6.E70)\), \([71](https://arxiv.org/html/2605.29267#A6.E71)\)\\displaystyle\\left\|\\int\_\{t=0\}^\{1\}\\left\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),\\left\(\-S\_\{p\}\\mathcal\{F\},\\frac\{t\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1\)\}\{t\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\}\}S\_\{p\}\\mathcal\{F\},\\frac\{t\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1\)\}\{t\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}\}S\_\{p\}\\mathcal\{F\}\\right\)\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1,\\widehat\{\\lambda\}\_\{\\mathcal\{S\}\}^\{\\theta\},\\widehat\{\\lambda\}\_\{\\mathcal\{H\}\}^\{\\theta\}\\right\)^\{T\}\\right\\rangle dt\\right\|\\ \\ \\ \\text\{by Eq\.~\(\\ref\{eq:paramGradRealSyn\}\), \(\\ref\{eq:FpFqGradRealSyn\}\)\}\(ℱ:=∂Fp∂λℛθ​\(λℛθ=1\+t​\(λ^ℛθ−1\)\)=𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]−𝔼P​\(θ∗,ϕ∗\)/ℛθ​\[∇θℓθ​\(θ∗\)\]by Eq\. \([71](https://arxiv.org/html/2605.29267#A6.E71)\)\)\\displaystyle\\left\(\\mathcal\{F\}:=\\frac\{\\partial F\_\{p\}\}\{\\partial\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}\}\\left\(\\lambda\_\{\\mathcal\{R\}\}^\{\\theta\}=1\+t\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\-1\)\\right\)=\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\-\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\ \\ \\ \\text\{by Eq\.~\(\\ref\{eq:FpFqGradRealSyn\}\)\}\\right\)=\\displaystyle=\\3​\(1−λ^ℛθ\)​\|∫t=01⟨∇θJp​\(θ∗\),Sp​ℱ⟩​𝑑t\|≤3​\(1−λ^ℛθ\)​supt∈\[0,1\]​\|⟨∇θJp​\(θ∗\),Sp​ℱ⟩\|\\displaystyle 3\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\left\|\\int\_\{t=0\}^\{1\}\\left\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\mathcal\{F\}\\right\\rangle dt\\right\|\\leq 3\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\underset\{t\\in\[0,1\]\}\{\\sup\}\\left\|\\left\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\mathcal\{F\}\\right\\rangle\\right\|=\\displaystyle=\\3​\(1−λ^ℛθ\)​supt∈\[0,1\]​\|⟨∇θJp​\(θ∗\),Sp​𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]−𝔼P​\(θ∗,ϕ∗\)/ℛθ​\[∇θℓθ​\(θ∗\)\]⟩\|\\displaystyle 3\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\underset\{t\\in\[0,1\]\}\{\\sup\}\\left\|\\left\\langle\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\),S\_\{p\}\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\-\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\right\\rangle\\right\|≤\\displaystyle\\leq\\3​\(1−λ^ℛθ\)​supt∈\[0,1\]​‖∇θJp​\(θ∗\)‖​‖Sp‖​‖𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]−𝔼P​\(θ∗,ϕ∗\)/ℛθ​\[∇θℓθ​\(θ∗\)\]‖\\displaystyle 3\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\underset\{t\\in\[0,1\]\}\{\\sup\}\\left\\lVert\\nabla\_\{\\theta\}J\_\{p\}\(\\theta^\{\*\}\)\\right\\rVert\\left\\lVert S\_\{p\}\\right\\rVert\\left\\lVert\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\-\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\right\\rVert≤\\displaystyle\\leq\\3​\(1−λ^ℛθ\)​supt∈\[0,1\]​L2​‖Sp‖​‖𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]−𝔼P​\(θ∗,ϕ∗\)/ℛθ​\[∇θℓθ​\(θ∗\)\]‖\\displaystyle 3\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\underset\{t\\in\[0,1\]\}\{\\sup\}L^\{2\}\\left\\lVert S\_\{p\}\\right\\rVert\\left\\lVert\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\-\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\right\\rVert\(72\)where⟨⟩\\langle\\rangledenotes the inner product between two vectors\. By the proof of Proposition[4\.4](https://arxiv.org/html/2605.29267#S4.Thmtheorem4)in Appendix[F\.2](https://arxiv.org/html/2605.29267#A6.SS2)and Assumptions[3\.2](https://arxiv.org/html/2605.29267#S3.Thmtheorem2)\-[3\.5](https://arxiv.org/html/2605.29267#S3.Thmtheorem5),Sp−1⪰\(γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\)​I≻0S\_\{p\}^\{\-1\}\\succeq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\)I\\succ 0, so∥Sp∥≤\(γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ\)−1\\lVert S\_\{p\}\\rVert\\leq\(\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\)^\{\-1\}\(the proof of Corollary[4\.7](https://arxiv.org/html/2605.29267#S4.Thmtheorem7)shows the details in Appendix[F\.6](https://arxiv.org/html/2605.29267#A6.SS6)\)\. Also, by Assumption[3\.3](https://arxiv.org/html/2605.29267#S3.Thmtheorem3),ℓθ\\ell\_\{\\theta\}isLθL\_\{\\theta\}\-Lipschitz inθ\\theta, so

∥𝔼ℛθ​\[∇θℓθ​\(θ∗\)\]−𝔼P​\(θ∗,ϕ∗\)/ℛθ​\[∇θℓθ​\(θ∗\)\]∥≤𝔼ℛθ​∥∇θℓθ​\(θ∗\)∥\+𝔼P​\(θ∗,ϕ∗\)/ℛθ​∥∇θℓθ​\(θ∗\)∥≤2​Lθ\.\\lVert\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\-\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\[\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\]\\rVert\\leq\\mathbb\{E\}\_\{\\mathcal\{R\}^\{\\theta\}\}\\lVert\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\rVert\+\\mathbb\{E\}\_\{P\(\\theta^\{\*\},\\phi^\{\*\}\)/\\mathcal\{R\}^\{\\theta\}\}\\lVert\\nabla\_\{\\theta\}\\ell\_\{\\theta\}\(\\theta^\{\*\}\)\\rVert\\leq 2L\_\{\\theta\}\.Therefore, for Eq\. \([72](https://arxiv.org/html/2605.29267#A6.E72)\), we can get

\|Jp​\(θ∗​\(λ^ℛθ\)\)−Jp​\(θ∗​\(1\)\)\|≤\(1−λ^ℛθ\)​6​Lθ​L2γθ−Lθ​εθ−Lθ​εθ​Lϕ​εϕγϕ−Lϕ​εϕ=𝒪​\(1−λ^ℛθ\)\.\\left\|J\_\{p\}\\left\(\\theta^\{\*\}\\left\(\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\\right\)\\right\)\-J\_\{p\}\\left\(\\theta^\{\*\}\\Big\(1\\Big\)\\right\)\\right\|\\leq\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\\frac\{6L\_\{\\theta\}L^\{2\}\}\{\\gamma\_\{\\theta\}\-L\_\{\\theta\}\\varepsilon\_\{\\theta\}\-\\frac\{L\_\{\\theta\}\\varepsilon\_\{\\theta\}L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\{\\gamma\_\{\\phi\}\-L\_\{\\phi\}\\varepsilon\_\{\\phi\}\}\}=\\mathcal\{O\}\(1\-\\widehat\{\\lambda\}\_\{\\mathcal\{R\}\}^\{\\theta\}\)\.\(73\)Similarly we can prove the same result forJqJ\_\{q\}\. ∎