$\mathbf{\lambda}$-VAE: Variance Equalization for Posterior Collapse

# Variance Equalization for Posterior Collapse
Source: [https://arxiv.org/html/2607.05531](https://arxiv.org/html/2607.05531)
###### Abstract

Variational Autoencoders \(VAEs\) frequently suffer from posterior collapse, a failure mode in which the approximate posterior converges to the prior, rendering the latent code uninformative\. Despite extensive research, a unified account of why collapse occurs has remained an open question\. We identify and formalize two logically independent but coupled causes\.*Gradient imbalance*occurs when the decoder’s reconstruction signal vanishes faster than the𝕂​𝕃\\mathbb\{KL\}regularization pressure as the posterior widens\.*Information gap*occurs when the stochastic sampling step discards a substantial fraction of the encoder’s computed representation, attenuating decoder sensitivity and making collapse inexpensive\. Both causes share the same collapse trajectory, and we show that the information gap is algebraically equivalent to mismatch between the aggregate posterior and the prior, unifying two pathologies\. Subsequently, we introduceλ\\lambda\-VAE, which resolves both causes through a single modification to the reparameterization step: the sampling noise is scaled by per\-dimension exponent, while the𝕂​𝕃\\mathbb\{KL\}penalty retains the original posterior variance\. This asymmetry shifts the stable training attractor away from the degenerate collapsed state, driving all latent dimensions toward the same equilibrium – a mechanism we term*variance equalization*\. A closed\-form optimal exponent per dimension follows from a net information gain objective, with a single hyperparameter controlling the reconstruction–generation tradeoff\. We validate on standard benchmarks \(Binary MNIST, Binary Omniglot, CIFAR\-10, CelebA\-64\), showing consistent reductions in collapsed dimensions, information capacity gains of up to2\.8×2\.8\\timesnats, and reconstruction quality improvements of up to\+0\.33\+0\.33BPD\.

## 1Introduction

Variational Autoencoders \(VAEs\)Kingma and Welling \([2013](https://arxiv.org/html/2607.05531#bib.bib1)\); Rezendeet al\.\([2014](https://arxiv.org/html/2607.05531#bib.bib2)\)have become a foundational tool for learning latent representations, with applications in image generationRazaviet al\.\([2019b](https://arxiv.org/html/2607.05531#bib.bib20)\), natural language processingBowmanet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib17)\), multimodal modellingShiet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib13)\), and semi\-supervised learningKingmaet al\.\([2014](https://arxiv.org/html/2607.05531#bib.bib39)\)\. VAE is one of many variational inference frameworks that use approximate posterior for maximizing data likelihoodNeal and Hinton \([1998](https://arxiv.org/html/2607.05531#bib.bib5)\); McLachlan and Krishnan \([2007](https://arxiv.org/html/2607.05531#bib.bib6)\); Minka \([2013](https://arxiv.org/html/2607.05531#bib.bib7)\); Jordanet al\.\([1999](https://arxiv.org/html/2607.05531#bib.bib8)\)\. It is, however, one of the first to use*amortization*to estimate approximate posterior; an encoder learns to map each input to a posterior distribution over a latent space, and a decoder reconstructs the input from a sample\. This is a highly efficient and scalable approach, but it is prone to a well\-documented failure mode called*posterior collapse*Bowmanet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib17)\); Chenet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib18)\); Lucaset al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib38)\), in which the learned approximate posterior converges to the prior partially or fully\. When collapse occurs, the decoder learns to reconstruct data from the prior\. In its extreme form, the mutual information between data and latent variables goes to zero \(I​\(X;Z\)≈0I\(X;Z\)\\approx 0\), and the latent space loses all its representational value\. This failure is most severe with high\-capacity decodersBowmanet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib17)\); Razaviet al\.\([2019b](https://arxiv.org/html/2607.05531#bib.bib20)\), such as those with autoregressive or deep convolutional architectures, but it occurs to varying degrees in most practical VAE training runs\.

Extensive research has proposed different solutions for posterior collapse; re\-weighting the Kullback\-Leibler \(𝕂​𝕃\\mathbb\{KL\}\) divergence termHigginset al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib29)\); Fuet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib31)\), imposing minimum𝕂​𝕃\\mathbb\{KL\}floorsChenet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib18)\); Razaviet al\.\([2019a](https://arxiv.org/html/2607.05531#bib.bib32)\), matching the aggregate posterior explicitlyZhaoet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib40)\); Tolstikhinet al\.\([2017](https://arxiv.org/html/2607.05531#bib.bib33)\), enriching the posterior familyRezende and Mohamed \([2015](https://arxiv.org/html/2607.05531#bib.bib26)\); Kingmaet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib53)\), and adjusting training dynamicsHeet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib24)\)\. These methods target different manifestations of the problem and share a common limitation: they act on the training objective and propose a global solution, while lacking a unified account on the causes of collapse\.

In this paper, we provide a formal account of why posterior collapse occurs and build a targeted solution on top of it\. Our starting observation is simple: in a standard VAE, the𝕂​𝕃\\mathbb\{KL\}term selectively pushes low\-signal dimensions towardσi=1\\sigma\_\{i\}=1while leaving high\-signal dimensions active, producing a*polarized*distribution of posterior variances\. This polarization is both the signature of emerging collapse and a driver of further collapse\. Hence, the right intervention is not to globally modify the training objective but to counteract the polarization directly per dimension, which we formalize and validate in subsequent sections\.

Summary of Contributions:

1. 1\.We prove that posterior collapse has two logically independent but coupled causes:*gradient imbalance*\(Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1)\), in which the reconstruction gradient vanishes before the𝕂​𝕃\\mathbb\{KL\}restoring force as the posterior widens, and*information gap*\(Proposition[2](https://arxiv.org/html/2607.05531#Thmprop2)\), in which the stochastic bottleneck discards encoder signal, making collapse inexpensive\. We further show that the information gap is algebraically equivalent to marginal mismatch \(difference between the aggregate posterior and the prior\) \(Eq\.[8](https://arxiv.org/html/2607.05531#S3.E8)\)\.
2. 2\.We proposeλ\\lambda\-VAE, replacing the reparameterization noiseσ​ϵ\\sigma\\epsilonwithσλ​ϵ\\sigma^\{\\lambda\}\\epsilonwhile retaining the𝕂​𝕃\\mathbb\{KL\}penalty on the originalσ2\\sigma^\{2\}resolving both causes simultaneously\. The asymmetry drives all latent dimensions toward the same equilibrium –*variance equalization*– without modifying the training objective or adding parameters\. A closed\-form optimal per\-dimension exponent follows from a net information gain objective \(Proposition[3](https://arxiv.org/html/2607.05531#Thmprop3)\), with a single hyperparameterδ\\deltacontrolling the reconstruction–generation tradeoff\.
3. 3\.We show that on binarized benchmarks,λ\\lambda\-VAE reduces collapsed dimensions from 16 to 1 on Binary MNIST and 13 to 0 on Binary Omniglot while improving reconstruction quality\. On RGB images, information capacity grows by up to2\.8×2\.8\\timesand BPD improves by\+0\.33\+0\.33on CIFAR\-10\. A PixelCNN experiment reveals a fundamental limitation of BPD as a collapse diagnostic: two models with near\-identical BPD \(3\.518 vs\. 3\.494\) differ by6\.2×6\.2\\timesin the decoder capacity allocated to the latent code\.

## 2Variational Autoencoders

VAEKingma and Welling \([2013](https://arxiv.org/html/2607.05531#bib.bib1)\); Rezendeet al\.\([2014](https://arxiv.org/html/2607.05531#bib.bib2)\)is one of the earlier approaches to use*amortization*and stochastic gradient estimation in large\-scale datasets via the Evidence Lower Bound \(ELBO\), written as

ℒ=𝔼qψ​\(z\|x\)​\[log⁡pθ​\(x\|z\)\]−𝕂​𝕃​\(qψ​\(z\|x\)∥p​\(z\)\),\\displaystyle\\mathcal\{L\}=\\mathbb\{E\}\_\{q\_\{\\psi\}\(z\|x\)\}\\\!\\left\[\\log p\_\{\\theta\}\(x\|z\)\\right\]\-\\mathbb\{KL\}\\\!\\left\(q\_\{\\psi\}\(z\|x\)\\,\\\|\\,p\(z\)\\right\),\(1\)withqψ​\(z\|x\)=𝒩​\(μ​\(x\),diag​\(σ2​\(x\)\)\)q\_\{\\psi\}\(z\|x\)=\\mathcal\{N\}\(\\mu\(x\),\\mathrm\{diag\}\(\\sigma^\{2\}\(x\)\)\),p​\(z\)=𝒩​\(0,𝐈\)p\(z\)=\\mathcal\{N\}\(0,\\mathbf\{I\}\), and sampling viaz=μ​\(x\)\+σ​\(x\)​ϵz=\\mu\(x\)\+\\sigma\(x\)\\epsilon,ϵ∼𝒩​\(0,𝐈\)\\epsilon\\sim\\mathcal\{N\}\(0,\\mathbf\{I\}\)\. The Gaussian assumption on the posterior and prior makes the𝕂​𝕃\\mathbb\{KL\}term computable in a closed form;𝕂​𝕃i=12​\(μi2\+σi2−log⁡σi2−1\)\\mathbb\{KL\}\_\{i\}=\\frac\{1\}\{2\}\(\\mu\_\{i\}^\{2\}\+\\sigma\_\{i\}^\{2\}\-\\log\\sigma\_\{i\}^\{2\}\-1\), henceforth we work per latent dimensioniiand suppress the subscript where unambiguous\. InHoffman and Johnson \([2016](https://arxiv.org/html/2607.05531#bib.bib27)\), the𝕂​𝕃\\mathbb\{KL\}term in ELBO is decomposed and the overall loss is written as

ℒ=𝔼​\[log⁡pθ​\(x\|z\)\]−Iq​\(X;Z\)−𝕂​𝕃​\(qψ​\(z\)∥p​\(z\)\),\\displaystyle\\mathcal\{L\}=\\mathbb\{E\}\\\!\\left\[\\log p\_\{\\theta\}\(x\|z\)\\right\]\-I\_\{q\}\(X;Z\)\-\\mathbb\{KL\}\(q\_\{\\psi\}\(z\)\\\|p\(z\)\),\(2\)whereIq​\(X;Z\)I\_\{q\}\(X;Z\)is the mutual information, and𝕂​𝕃​\(qψ​\(z\)∥p​\(z\)\)\\mathbb\{KL\}\(q\_\{\\psi\}\(z\)\\\|p\(z\)\)is the marginal mismatch\. Hence, minimizing the𝕂​𝕃\\mathbb\{KL\}term in Eq\.[1](https://arxiv.org/html/2607.05531#S2.E1)reduces bothIq​\(X;Z\)I\_\{q\}\(X;Z\)and the marginal mismatch; the optimizer cannot distinguish between them, which is why collapse is an unintended consequence\. That is, at𝕂​𝕃≈0\\mathbb\{KL\}\\approx 0in Eq\.[1](https://arxiv.org/html/2607.05531#S2.E1), the latent codezzbecomes statistically independent of the inputxx, rendering the encoder information useless in the signal reconstruction\.

In general, ELBO creates an information bottleneck that encourages encoding the least amount of information subject to reconstruction errorTishbyet al\.\([2000](https://arxiv.org/html/2607.05531#bib.bib61)\)\. Posterior collapse, however, is an extreme case in which the model achieves sufficient reconstruction accuracy while ignoring all input signal,Iq​\(X;Z\)≈0I\_\{q\}\(X;Z\)\\approx 0– often observed and measured per latent dimension\. In Section[3](https://arxiv.org/html/2607.05531#S3), we will expand on both data and model specific conditions that enable VAE training to find solutions that fail to encode details about the input while achieving high reconstruction accuracy\.

## 3Posterior Collapse: Two Causes

Posterior collapse in trained models manifests as near\-zero per\-dimension𝕂​𝕃\\mathbb\{KL\}, a low count of active latent dimensions, andIq​\(X;Z\)≈0I\_\{q\}\(X;Z\)\\approx 0\. In the subsequent sections, we present two principal causes for posterior collapse and show their relationship to𝕂​𝕃\\mathbb\{KL\}and mutual information\.

### 3\.1Gradient Imbalance

Differentiating Eq\.[1](https://arxiv.org/html/2607.05531#S2.E1)with respect toσi\\sigma\_\{i\}via the reparameterisationzi=μi\+σi​ϵiz\_\{i\}=\\mu\_\{i\}\+\\sigma\_\{i\}\\epsilon\_\{i\}, we have

∂ℒ∂σi=𝔼p​\(x\),ϵi​\[∇zilog⁡pθ​\(x\|z\)⋅ϵi\]\+1σi−σi\.\\displaystyle\\frac\{\\partial\\mathcal\{L\}\}\{\\partial\\sigma\_\{i\}\}=\\mathbb\{E\}\_\{p\(x\),\\,\\epsilon\_\{i\}\}\\\!\\bigl\[\\nabla\_\{z\_\{i\}\}\\log p\_\{\\theta\}\(x\|z\)\\cdot\\epsilon\_\{i\}\\bigr\]\+\\frac\{1\}\{\\sigma\_\{i\}\}\-\\sigma\_\{i\}\.\(3\)The𝕂​𝕃\\mathbb\{KL\}restoring force \(second term\) is positive forσi<1\\sigma\_\{i\}<1, zero atσi=1\\sigma\_\{i\}=1, and negative forσi\>1\\sigma\_\{i\}\>1\. As such, it always drivesσi\\sigma\_\{i\}toward 1 while looking for a stable solution in Eq\.[3](https://arxiv.org/html/2607.05531#S3.E3)\. When the first term, the reconstruction gradient which we denote bygiσg\_\{i\}^\{\\sigma\}, vanishes thenσi=1\\sigma\_\{i\}=1is the sole equilibrium\. Hence, the dynamics betweengiσg\_\{i\}^\{\\sigma\}and the𝕂​𝕃\\mathbb\{KL\}restoring force determine whether a dimension remains active or collapses during training\.

We define the*gradient ratio*ρi=\|giσ\|/\|1/σi−σi\|\\rho\_\{i\}=\|g\_\{i\}^\{\\sigma\}\|/\|1/\\sigma\_\{i\}\-\\sigma\_\{i\}\|as the primary indicator of collapse\. That is, whenρi≪1\\rho\_\{i\}\\ll 1, the𝕂​𝕃\\mathbb\{KL\}term dominates and the dimension collapses\. The following proposition formalizes this understanding\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x1.png)\(a\)Gradient imbalance \(ρi\\rho\_\{i\}vsσi\\sigma\_\{i\}\)
![Refer to caption](https://arxiv.org/html/2607.05531v1/x2.png)\(b\)Information gap \(Δ​I\\Delta I,I​\(X;Z\)I\(X;Z\),ccvsσi\\sigma\_\{i\}\)

Figure 1:Posterior collapse causes\.*\(a\)*Gradient ratioρi\\rho\_\{i\}under the decay modelgiσ=g0​\(1−σi\)αg\_\{i\}^\{\\sigma\}=g\_\{0\}\(1\-\\sigma\_\{i\}\)^\{\\alpha\},g0=2g\_\{0\}=2\. Forα\>1\\alpha\>1\(Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1)\),ρi→0\\rho\_\{i\}\\to 0asσi→1\\sigma\_\{i\}\\to 1: the reconstruction signal vanishes faster than the𝕂​𝕃\\mathbb\{KL\}restoring force, making the collapsed state the only stable fixed point\. Forα<1\\alpha<1,ρi\\rho\_\{i\}is no longer a reliable indicator: it stays above 1 even when the dimension has collapsed, because the reconstruction gradient from the decoder decays much more slowly in this regime\.*\(b\)*Mutual informationI​\(X;Z\)I\(X;Z\), information gapΔ​I\\Delta I, and attenuation factorc=I​\(X;Z\)/I​\(X;ϕ\)c=I\(X;Z\)/I\(X;\\phi\)using the Gaussian channel model withSi=1\.5S\_\{i\}=1\.5\. Asσi\\sigma\_\{i\}increases towards 1:I​\(X;Z\)I\(X;Z\)falls,Δ​I\\Delta Igrows towardI​\(X;ϕ\)I\(X;\\phi\), andc→0c\\to 0\. Hence, collapse becomes much cheaper \(Proposition[2](https://arxiv.org/html/2607.05531#Thmprop2)\)\.###### Proposition 1\(Gradient Imbalance\)\.

Suppose the asymptotic decay condition for the reconstruction gradient is\|giσ\|=O​\(\(1−σi\)α\)\|g\_\{i\}^\{\\sigma\}\|=O\(\(1\-\\sigma\_\{i\}\)^\{\\alpha\}\)asσi→1−\\sigma\_\{i\}\\to 1^\{\-\}withα\>1\\alpha\>1\. Then dimensioniicollapses to the prior if and only iflim inft→∞ρi​\(t\)<ρcrit\\liminf\_\{t\\to\\infty\}\\rho\_\{i\}\(t\)<\\rho\_\{\\mathrm\{crit\}\}for some finiteρcrit\>0\\rho\_\{\\mathrm\{crit\}\}\>0\.111The idealized gradient\-balance value isρcrit=1\\rho\_\{\\mathrm\{crit\}\}=1\(any non\-collapsed equilibrium of∂ℒ/∂σi\\partial\\mathcal\{L\}/\\partial\\sigma\_\{i\}satisfiesρi≥1\\rho\_\{i\}\\geq 1\)\. In practice, joint encoder–decoder optimization shifts the effective threshold; on CIFAR\-10, for instance, the empiricalρcrit≈0\.006\\rho\_\{\\mathrm\{crit\}\}\\approx 0\.006separates active from collapsed dimensions with 99\.8% accuracy\. The value 1 therefore bounds the theoretical regime; the operative threshold is model and dataset specific\.

Under this condition,ρi→0\\rho\_\{i\}\\to 0asσi→1\\sigma\_\{i\}\\to 1, makingσi=1\\sigma\_\{i\}=1the sole stable fixed point\. The decay exponentα\\alphacharacterizes the decoder’s sensitivity to the latent code near collapse\. The conditionα\>1\\alpha\>1identifies decoders for whichρi\\rho\_\{i\}is a necessary and sufficient collapse signal\. Autoregressive decoders satisfy this withα≫1\\alpha\\gg 1, explaining their high collapse rateBowmanet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib17)\)and empirically confirmed by the PixelCNN experiment in Section[6\.4](https://arxiv.org/html/2607.05531#S6.SS4)\. The collapsed fixed pointσi∗=1\\sigma\_\{i\}^\{\*\}=1is locally stable with linearization rate−2\-2; by contrast,μi∗=0\\mu\_\{i\}^\{\*\}=0has rate−1\-1, which is whyσi→1\\sigma\_\{i\}\\to 1is the more reliable leading indicator of collapse\. Full proof and details are in Appendix[A](https://arxiv.org/html/2607.05531#A1.SSx1), while illustration of the dynamics is shown in Figure[1\(a\)](https://arxiv.org/html/2607.05531#S3.F1.sf1)\.

### 3\.2Information Gap

The encoder in a VAE, which estimates the posterior parameters, is a deterministic learned mapϕ​\(x\)=\(μ​\(x\),log⁡σ​\(x\)\)\\phi\(x\)=\(\\mu\(x\),\\log\\sigma\(x\)\), while the latentZZis obtained by stochastic reparameterization\. This defines a Markov chainX→ϕ​\(X\)→ZX\\to\\phi\(X\)\\to Z, for which the data processing inequalityCover \([1999](https://arxiv.org/html/2607.05531#bib.bib62)\)gives:

H​\[X\]≥I​\(X;ϕ\)≥I​\(X;Z\)≥0,\\displaystyle H\[X\]\\;\\geq\\;I\(X;\\phi\)\\;\\geq\\;I\(X;Z\)\\;\\geq\\;0,\(4\)whereH​\(⋅\)H\(\\cdot\)denotes entropy of a random variable\. SinceI​\(X;ϕ\)=H​\[ϕ\]I\(X;\\phi\)=H\[\\phi\]\(encoder is deterministic\), the*information gap*is the information capacity that the sampling step loses, captured as

Δ​I=H​\[ϕ\]−I​\(X;Z\)≥0\.\\displaystyle\\Delta I\\;=\\;H\[\\phi\]\-I\(X;Z\)\\;\\geq\\;0\.\(5\)The information gap widens if the dataset has low signal power \(low variance inμi​\(X\)\\mu\_\{i\}\(X\)\), which reducesI​\(X;Z\)I\(X;Z\)asσ→1\\sigma\\to 1\. More importantly, a large gap implies an attenuation factor that dampens decoder sensitivity to encoder change, facilitating collapse\. The following proposition formalizes this\.

###### Proposition 2\(Information Gap\)\.

Letℛ=f​\(I​\(X;Z\)\)\\mathcal\{R\}=f\(I\(X;Z\)\)be reconstruction quality measure for monotonically increasingff\. The sensitivity of reconstruction quality to encoder capacity is attenuated as

∂ℛ∂I​\(X;ϕ\)=∂ℛ∂I​\(X;Z\)⋅\(1−∂Δ​I∂I​\(X;ϕ\)\)\.\\displaystyle\\frac\{\\partial\\mathcal\{R\}\}\{\\partial I\(X;\\phi\)\}=\\frac\{\\partial\\mathcal\{R\}\}\{\\partial I\(X;Z\)\}\\cdot\\left\(1\-\\frac\{\\partial\\Delta I\}\{\\partial I\(X;\\phi\)\}\\right\)\.\(6\)

The attenuation factorc=1−∂Δ​I/∂I​\(X;ϕ\)≈I​\(X;Z\)/I​\(X;ϕ\)c=1\-\\partial\\Delta I/\\partial I\(X;\\phi\)\\approx I\(X;Z\)/I\(X;\\phi\)\(under the approximation that the two move proportionally during training\) measures how much of the encoder’s signal the decoder can access\.

The Gaussian channel model makes the information gap view concrete by treatingzi=μi​\(X\)\+σi​ϵiz\_\{i\}=\\mu\_\{i\}\(X\)\+\\sigma\_\{i\}\\epsilon\_\{i\}as transmitting signalμi\\mu\_\{i\}through noiseσi​ϵi\\sigma\_\{i\}\\epsilon\_\{i\}, giving

I​\(X;Zi\)≈12​log⁡\(1\+Siσi2\),\\displaystyle I\(X;Z\_\{i\}\)\\;\\approx\\;\\tfrac\{1\}\{2\}\\log\\\!\\left\(1\+\\frac\{S\_\{i\}\}\{\\sigma\_\{i\}^\{2\}\}\\right\),\(7\)whereSi=Var​\[μi​\(X\)\]S\_\{i\}=\\mathrm\{Var\}\[\\mu\_\{i\}\(X\)\]is the signal power\. High SNR \(signal\-to\-noise ratio\) givesc≈1c\\approx 1, making collapse costly; low SNR givesc→0c\\to 0and collapse exerts zero pressure on the decoder – it is free \(Figure[1\(b\)](https://arxiv.org/html/2607.05531#S3.F1.sf2)\)\. More importantly, under marginal matching whereq​\(z\)≈𝒩​\(0,𝐈\)q\(z\)\\approx\\mathcal\{N\}\(0,\\mathbf\{I\}\)the signal powerSi≈1−σi2S\_\{i\}\\approx 1\-\\sigma\_\{i\}^\{2\}, this reduces toI​\(X;Zi\)≈−log⁡σiI\(X;Z\_\{i\}\)\\approx\-\\log\\sigma\_\{i\}as a bound to the accessible information which goes to zero asσ→1\\sigma\\to 1; result is further discussed in Appendix[A](https://arxiv.org/html/2607.05531#A1.SSx2)\.

Overall, the information gap decomposes into a reducible and an irreducible part\. The irreducible floor is set by the data’s intrinsic dimensionality: dimensions where the data provides no signal which contribute an unavoidable gap regardless of encoder quality\. For instance, in binarized MNIST and Omniglot, corner pixels carry no information; when encoded into a high\-dimensional latent space, the corresponding dimensions will naturally collapse\. The reducible part is𝕂​𝕃\\mathbb\{KL\}\-induced; since𝕂​𝕃\\mathbb\{KL\}forcesSi≈1−σi2S\_\{i\}\\approx 1\-\\sigma\_\{i\}^\{2\}, any rise inσi\\sigma\_\{i\}under regularization pressure directly suppressesSiS\_\{i\}, creating an information gap a higher\-SNR encoder could avoid\. Intuitively, a lower information gap implies a more distributed representation – all dimensions carry useful signal, ideally independent rather than redundant – whereas a sparse representation concentrates signal in few overloaded dimensions and leaves the rest high\-noise, seeHinton \([1986](https://arxiv.org/html/2607.05531#bib.bib41)\); Bengioet al\.\([2013](https://arxiv.org/html/2607.05531#bib.bib42)\)\.

A direct consequence follows from the ELBO decompositionHoffman and Johnson \([2016](https://arxiv.org/html/2607.05531#bib.bib27)\)\. SubstitutingI​\(X;Z\)=I​\(X;ϕ\)−Δ​II\(X;Z\)=I\(X;\\phi\)\-\\Delta Iinto Eq\.[2](https://arxiv.org/html/2607.05531#S2.E2):

𝕂​𝕃​\(qψ​\(z\)∥p​\(z\)\)=𝕂​𝕃​\(qψ​\(z\|x\)∥p​\(z\)\)−I​\(X;ϕ\)\+Δ​I\.\\displaystyle\\mathbb\{KL\}\(q\_\{\\psi\}\(z\)\\\|p\(z\)\)=\\mathbb\{KL\}\(q\_\{\\psi\}\(z\|x\)\\\|p\(z\)\)\-I\(X;\\phi\)\+\\Delta I\.\(8\)This identity holds at every training snapshot\. Any increase inΔ​I\\Delta Ithat is not compensated by a decrease in𝕂​𝕃​\(q​\(z\|x\)∥p​\(z\)\)−I​\(X;ϕ\)\\mathbb\{KL\}\(q\(z\|x\)\\\|p\(z\)\)\-I\(X;\\phi\)manifests directly as growing marginal mismatch\. Encoder attenuation and aggregate posterior misalignment are therefore algebraically equivalent: two descriptions of the same quantity, one from the sampling bottleneck and one from the aggregate distribution\. Hence, the information gap serves as a cheap, per\-batch proxy for marginal mismatch\.

### 3\.3Coupling Between the Two Causes

The two causes are logically independent; gradient imbalance depends on decoder architecture \(α\\alpha, and decoder capacity\); the information gap depends on data complexity \(SiS\_\{i\}\) and posterior width \(σi\\sigma\_\{i\}\)\. Either can occur without the other\. However, both are driven byσi→1\\sigma\_\{i\}\\to 1: gradient imbalance makes this trajectory dynamically inevitable onceρi<ρcrit\\rho\_\{i\}<\\rho\_\{\\mathrm\{crit\}\}under the asymptotic conditionα\>1\\alpha\>1; the information gap makes reachingσi=1\\sigma\_\{i\}=1less costly before that point\.

Once gradient imbalance pushesρi\\rho\_\{i\}below the critical point,σi\\sigma\_\{i\}begins to rise\. For data with low latent complexitySiS\_\{i\}, the risingσi\\sigma\_\{i\}opens a substantial information gap, lowers the attenuation factorcc, and weakens the reconstruction gradientgiσg\_\{i\}^\{\\sigma\}which reducesρi\\rho\_\{i\}further and accelerates the rise inσi\\sigma\_\{i\}\. The feedback cycle is closed and positive; gradient imbalance provides the initial push, and the information gap amplifies it when the encoder has high capacity\. On complex data, the information gap remains shallow and the attenuation is weak; collapse is then driven primarily by gradient imbalance alone\. In both cases, however, the polarization ofσ\\sigmais the signature of the trajectory toward collapse, and the key point of intervention\. In Section[4](https://arxiv.org/html/2607.05531#S4), we present an approach that actively counteracts this polarization, addressing both causes simultaneously\.

## 4λ\\lambda\-VAE: Variance Equalization

Consider the modification of the latent sampling step in ELBO as

z=μ​\(x\)\+σ​\(x\)λ⋅ϵ,λ≥1,\\displaystyle z=\\mu\(x\)\+\\sigma\(x\)^\{\\lambda\}\\cdot\\epsilon,\\qquad\\lambda\\geq 1,\(9\)while computing the𝕂​𝕃\\mathbb\{KL\}term using the original varianceσ​\(x\)2\\sigma\(x\)^\{2\}\. Subsequently, the objective will be modified as

ℒλ=𝔼qλ​\(z\|x\)​\[log⁡pθ​\(x\|z\)\]−𝕂​𝕃​\(qψ​\(z\|x\)∥p​\(z\)\),\\displaystyle\\mathcal\{L\}\_\{\\lambda\}=\\mathbb\{E\}\_\{q\_\{\\lambda\}\(z\|x\)\}\\\!\\left\[\\log p\_\{\\theta\}\(x\|z\)\\right\]\-\\mathbb\{KL\}\\\!\\left\(q\_\{\\psi\}\(z\|x\)\\,\\\|\\,p\(z\)\\right\),\(10\)whereqλ​\(z\|x\)=𝒩​\(μ​\(x\),diag​\(σ​\(x\)2​λ\)\)q\_\{\\lambda\}\(z\|x\)=\\mathcal\{N\}\(\\mu\(x\),\\mathrm\{diag\}\(\\sigma\(x\)^\{2\\lambda\}\)\)is the sampling distribution but the𝕂​𝕃\\mathbb\{KL\}penalty usesσ​\(x\)2\\sigma\(x\)^\{2\}and notσ​\(x\)2​λ\\sigma\(x\)^\{2\\lambda\}; this isλ\\lambda\-VAE\.

λ\\lambda\-VAE introduces an asymmetry between the sampling and penalty terms\. The decoder receives noiseσiλ​ϵ\\sigma\_\{i\}^\{\\lambda\}\\epsilon\(reduced relative toσi​ϵ\\sigma\_\{i\}\\epsilonforσi<1\\sigma\_\{i\}<1,λ\>1\\lambda\>1\) while the𝕂​𝕃\\mathbb\{KL\}penalty evaluates the originalσi2\\sigma\_\{i\}^\{2\}\. This decoupling allows more information to flow through the bottleneck without inflating the𝕂​𝕃\\mathbb\{KL\}penalty\.

### 4\.1Gradient Rebalancing

The ELBO gradient with respect toσi\\sigma\_\{i\}underλ\\lambda\-scaling is

∂ℒλ∂σi=λ​σiλ−1⋅giσ\+1σi−σi\.\\displaystyle\\frac\{\\partial\\mathcal\{L\}\_\{\\lambda\}\}\{\\partial\\sigma\_\{i\}\}=\\lambda\\,\\sigma\_\{i\}^\{\\lambda\-1\}\\cdot g\_\{i\}^\{\\sigma\}\+\\frac\{1\}\{\\sigma\_\{i\}\}\-\\sigma\_\{i\}\.\(11\)Hence, the reconstruction term is amplified byλ​σiλ−1\\lambda\\sigma\_\{i\}^\{\\lambda\-1\}\. As a result, the effective gradient ratio becomesρiλ=λ​σiλ−1​ρi\\rho\_\{i\}^\{\\lambda\}=\\lambda\\sigma\_\{i\}^\{\\lambda\-1\}\\rho\_\{i\}\. Asσi→1\\sigma\_\{i\}\\to 1,ρiλ→λ​ρi\\rho\_\{i\}^\{\\lambda\}\\to\\lambda\\rho\_\{i\}\. A dimension withρi<ρcrit\\rho\_\{i\}<\\rho\_\{\\mathrm\{crit\}\}\(which would collapse underλ=1\\lambda=1\) is protected onceλ\>ρcrit/ρi\\lambda\>\\rho\_\{\\mathrm\{crit\}\}/\\rho\_\{i\}\.

Furthermore, a givenλi\\lambda\_\{i\}defines an attractorσi\\sigma\_\{i\}at which point the amplification vanishes\. The mechanism drivesσ\\sigmaabove or below the stable point towards it; see Figure[2\(a\)](https://arxiv.org/html/2607.05531#S4.F2.sf1)and Appendix[B\.1 Equilibrium Variance Targetσ∗=λ−1/\(λ−1\)\\sigma^\{\*\}=\\lambda^\{\-1/\(\\lambda\-1\)\}](https://arxiv.org/html/2607.05531#Ax1.SSx1)\. This behaviour leads to variance equalization across dimensions, which is the key mechanism ofλ\\lambda\-VAE to counteract*polarization*and hence posterior collapse\.

### 4\.2Information Gain

Underλ\\lambda\-scaling, usingH​\[Z\]≈H​\[Zλ\]H\[Z\]\\approx H\[Z\_\{\\lambda\}\]\(valid for moderateλ\\lambda; Appendix[A](https://arxiv.org/html/2607.05531#A1.SSx2)\), we have the following information gain

I​\(X;Zλ\)−I​\(X;Z1\)=λ−12​∑i=1d𝔼​\[−log⁡σi2\]≥0\.\\displaystyle I\(X;Z\_\{\\lambda\}\)\-I\(X;Z\_\{1\}\)=\\frac\{\\lambda\-1\}\{2\}\\sum\_\{i=1\}^\{d\}\\mathbb\{E\}\\\!\\left\[\-\\log\\sigma\_\{i\}^\{2\}\\right\]\\geq 0\.\(12\)All terms are positive forσi<1\\sigma\_\{i\}<1, giving a residual information gapΔ​Iλ=Δ​I1⋅\(2−λ\)\\Delta I\_\{\\lambda\}=\\Delta I\_\{1\}\\cdot\(2\-\\lambda\)\. As such, for a given training snapshot, atλ=2\\lambda=2the gap is eliminated, and forλ∈\(1,2\)\\lambda\\in\(1,2\)it is partially closed\. The information gain per dimension is proportional to𝔼​\[−log⁡σi2\]\\mathbb\{E\}\[\-\\log\\sigma\_\{i\}^\{2\}\], and thus confident dimensions \(smallσi\\sigma\_\{i\}\) yield larger information gains than dimensions with wide posterior\.

### 4\.3Variance Equalization and the Equilibrium Target

Consider the gradient correction introduced byλ\\lambda\-scaling, except this time with respect tolog⁡σi\\log\\sigma\_\{i\}; written as∇i=\(λ​σiλ−σi\)⋅giσ\\nabla\_\{i\}=\(\\lambda\\sigma\_\{i\}^\{\\lambda\}\-\\sigma\_\{i\}\)\\cdot g\_\{i\}^\{\\sigma\}\. Setting the gradient∇i=0\\nabla\_\{i\}=0, gives the equilibrium conditionλ​σiλ−1=1\\lambda\\sigma\_\{i\}^\{\\lambda\-1\}=1which is solved by

σi∗=λ−1/\(λ−1\)\.\\displaystyle\\sigma\_\{i\}^\{\*\}=\\lambda^\{\-1/\(\\lambda\-1\)\}\.\(13\)Full derivation and stability proof in Appendix[B\.1 Equilibrium Variance Targetσ∗=λ−1/\(λ−1\)\\sigma^\{\*\}=\\lambda^\{\-1/\(\\lambda\-1\)\}](https://arxiv.org/html/2607.05531#Ax1.SSx1)\. This value is the same regardless of starting initial posterior variance\. Hence, every dimension, whether near\-collapsed or over\-sharp, is drawn toward the sameσ∗\\sigma^\{\*\}, provided they share the sameλ\\lambda\. This is the content of*variance equalization*\. In addition, the attractor defined by Eq\.[13](https://arxiv.org/html/2607.05531#S4.E13)is stable\. That is, forσi\>σ∗\\sigma\_\{i\}\>\\sigma^\{\*\},∇i<0\\nabla\_\{i\}<0\(system pushesσi\\sigma\_\{i\}down\); forσi<σ∗\\sigma\_\{i\}<\\sigma^\{\*\},∇i\>0\\nabla\_\{i\}\>0\(system pushesσi\\sigma\_\{i\}up\)\. The correction is a bidirectional stable restoring force\. Furthermore, the linearization rate is1−λ1\-\\lambdaforλ\>1\\lambda\>1, so convergence is exponentially fast with rate\|λ−1\|\|\\lambda\-1\|\. Figure[2\(a\)](https://arxiv.org/html/2607.05531#S4.F2.sf1)illustrates the targetσ∗​\(λ\)\\sigma^\{\*\}\(\\lambda\)and convergence from multiple starting points\.

Hence, the polarizedσ\\sigmadistribution that characterizes emergent collapse is corrected simultaneously across all dimensions\. Similarly, the variance equalization leads to emergent distributed representation as opposed to sparse where entropy of the approximate posterior is maximizedHinton \([1986](https://arxiv.org/html/2607.05531#bib.bib41)\); Bengioet al\.\([2013](https://arxiv.org/html/2607.05531#bib.bib42)\)\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x3.png)\(a\)Convergence toσ∗\\sigma^\{\*\}
![Refer to caption](https://arxiv.org/html/2607.05531v1/x4.png)\(b\)Optimalλi∗​\(σi,δ\)\\lambda^\{\*\}\_\{i\}\(\\sigma\_\{i\},\\delta\)

Figure 2:Variance equalization inλ\\lambda\-VAE\.*\(a\)*Convergence dynamics forλ=2\\lambda=2\(σ∗=0\.5\\sigma^\{\*\}=0\.5\): six dimensions starting atσ0∈\{0\.10,0\.20,0\.35,0\.65,0\.80,0\.90\}\\sigma\_\{0\}\\in\\\{0\.10,0\.20,0\.35,0\.65,0\.80,0\.90\\\}all converge toσ∗=0\.5\\sigma^\{\*\}=0\.5\. Trajectories are solutions ofd​σ/d​t=σ−λ​σλd\\sigma/dt=\\sigma\-\\lambda\\sigma^\{\\lambda\}\(the gradient correction term with fixed reconstruction signal\), isolating the equalization mechanism\.*\(b\)*Optimal scaling exponentλi∗​\(σi,δ\)\\lambda^\{\*\}\_\{i\}\(\\sigma\_\{i\},\\delta\)forδ∈\{2,3,5,10\}\\delta\\in\\\{2,3,5,10\\\}\. Belowσc=1−1/δ\\sigma\_\{c\}=\\sqrt\{1\-1/\\delta\},λ∗=1\\lambda^\{\*\}=1; above it,λ∗\\lambda^\{\*\}increases withσi\\sigma\_\{i\}, allocating larger corrections to the most uncertain dimensions\.
### 4\.4Optimalλ∗\\lambda^\{\*\}per Dimension

Increasingλ\\lambdaimproves both gradient ratios and information gain but risks divergence from the prior, which is critical for random generation\. The𝕂​𝕃\\mathbb\{KL\}divergence from𝒩​\(0,𝐈\)\\mathcal\{N\}\(0,\\mathbf\{I\}\)under theλ\\lambda\-scaling is

𝕂​𝕃​\(λi\)=12​\(σi2​λi−σi2−log⁡σi2​λi\+log⁡σi2\)\>0for​λi\>1,σi<1\.\\displaystyle\\mathbb\{KL\}\(\\lambda\_\{i\}\)=\\frac\{1\}\{2\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-\\sigma\_\{i\}^\{2\}\-\\log\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\+\\log\\sigma\_\{i\}^\{2\}\\right\)\>0\\quad\\text\{for \}\\lambda\_\{i\}\>1,\\sigma\_\{i\}<1\.\(14\)As a result, we define an objective that can be used to calculate optimalλ\\lambdasuch that the information gain is balanced with the𝕂​𝕃\\mathbb\{KL\}divergence, as follows

𝒥​\(λi,σi\)=\(λi−1\)​\|log⁡σi\|−δ​𝕂​𝕃​\(λi\),δ\>1\.\\displaystyle\\mathcal\{J\}\(\\lambda\_\{i\},\\sigma\_\{i\}\)=\(\\lambda\_\{i\}\-1\)\|\\\!\\log\\sigma\_\{i\}\|\-\\delta\\,\\mathbb\{KL\}\(\\lambda\_\{i\}\),\\qquad\\delta\>1\.\(15\)The first term is the per\-dimension gain from Eq\.[12](https://arxiv.org/html/2607.05531#S4.E12); the second term is used to penalize the additional𝕂​𝕃\\mathbb\{KL\}divergence due toλi\\lambda\_\{i\}; it is weighted by a hyperparameterδ\>1\\delta\>1controlling the tradeoff\. The objective is strictly concave inλi\\lambda\_\{i\}and has a unique maximum, which is given by the following proposition\.

###### Proposition 3\(Optimalλ∗\\lambda^\{\*\}\)\.

The objective𝒥\\mathcal\{J\}is strictly concave inλi\\lambda\_\{i\}\. Its unconstrained maximum, subject toλi≥1\\lambda\_\{i\}\\geq 1, is:

λi∗=max⁡\(1,log⁡\(1−1/δ\)2​log⁡σi\)\.\\displaystyle\\lambda^\{\*\}\_\{i\}=\\max\\\!\\left\(1,\\;\\frac\{\\log\(1\-1/\\delta\)\}\{2\\log\\sigma\_\{i\}\}\\right\)\.\(16\)Full derivation in Appendix[B\.2 Optimal Per\-Dimensionλi∗\\lambda^\{\*\}\_\{i\}](https://arxiv.org/html/2607.05531#Ax1.SSx2)\.

The optimal solution provided by the above proposition has a direct interpretation as*optimal resource allocation*\. The estimateλi∗∝1/\|log⁡σi\|\\lambda^\{\*\}\_\{i\}\\propto 1/\|\\log\\sigma\_\{i\}\|, hence uncertain dimensions \(largeσi\\sigma\_\{i\}, small\|log⁡σi\|\|\\log\\sigma\_\{i\}\|\) receive larger corrections, while confident dimensions receive smaller ones or none at all\.

Meanwhile, the hyperparameterδ\\deltaplays a key role in determining the aggressiveness of correction regime\. Importantly, it determines a critical thresholdσc\\sigma\_\{c\}as

σc:=1−1/δ\\displaystyle\\sigma\_\{c\}:=\\sqrt\{1\-1/\\delta\}\(17\)For anyσi<σc\\sigma\_\{i\}<\\sigma\_\{c\},λ∗=1\\lambda^\{\*\}=1, the𝕂​𝕃\\mathbb\{KL\}divergence outweighs the gains, hence the max in Eq\.[16](https://arxiv.org/html/2607.05531#S4.E16)passes one\. The largerδ\\delta, the less tolerant the system is of𝕂​𝕃\\mathbb\{KL\}divergence\. In the extreme case, asδ→∞\\delta\\to\\inftythenλi∗→1\\lambda^\{\*\}\_\{i\}\\to 1for all dimensions, recovering the standard VAE\. On the contrary, asδ→1\+\\delta\\to 1^\{\+\}, the system grows more tolerant to disparity in𝕂​𝕃\\mathbb\{KL\}; approaching a deterministic autoencoder, see Figure[2\(b\)](https://arxiv.org/html/2607.05531#S4.F2.sf2)for this mechanism\.

## 5Related Works

Posterior collapse in VAEs has motivated research along three lines: modifying the𝕂​𝕃\\mathbb\{KL\}term or constraining decoder capacity to address gradient imbalance; isolating and minimizing marginal mismatch to improve aggregate posterior alignment; and enriching the prior or posterior family\. We discuss how each relates toλ\\lambda\-VAE\.

Inβ\\beta\-VAEHigginset al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib29)\); Burgesset al\.\([2018](https://arxiv.org/html/2607.05531#bib.bib30)\)ELBO is modified by weighting the𝕂​𝕃\\mathbb\{KL\}term\. Largeβ\>1\\beta\>1encourages disentanglement but can encourage the decoder not to use latent information early in training, leading to posterior collapse\. InFuet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib31)\), cyclical annealing ofβ\\betais introduced between near zero and its maximum value\. In both cases, the training objective is modified globally, affecting both reconstruction and𝕂​𝕃\\mathbb\{KL\}gradients symmetrically, while offering no per\-dimension adaptivity\. Free BitsChenet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib18)\)imposes a per\-dimension minimum𝕂​𝕃\\mathbb\{KL\}floor, introducing discontinuous gradients but guaranteeing minimum information usage\.δ\\delta\-VAERazaviet al\.\([2019a](https://arxiv.org/html/2607.05531#bib.bib32)\)enforces a similar constraint differentiably, leaving the threshold per dimension to be tuned\. None of the above approaches introduce adaptive control over information flow conditioned on local encoder confidenceσi\\sigma\_\{i\}as is done inλ\\lambda\-VAE\. Regardless, sinceλ\\lambda\-VAE modifies the sampling step rather than the training objective, it is fully compatible withβ\\beta\-VAE, cyclical annealing, and other𝕂​𝕃\\mathbb\{KL\}\-weighting schedules and can be composed freely\.

Normalising flowsRezende and Mohamed \([2015](https://arxiv.org/html/2607.05531#bib.bib26)\); Kingmaet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib53)\)enrich the posterior family through invertible transformations, reducing the approximation gap at the cost of significant parameter overhead\. Rosca et al\.Roscaet al\.\([2018](https://arxiv.org/html/2607.05531#bib.bib15)\)show that even expressive posteriors do not guarantee good aggregate matching, consistent with our finding that the information gap and marginal mismatch can be tracked to the same underlying issue\.

Lagging inferenceHeet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib24)\)updates the encoder more aggressively than the decoder to prevent early collapse, targeting gradient imbalance but not the information gap or marginal mismatch\. Linear VAE analysesLucaset al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib38)\); Ichikawa and Hukushima \([2024](https://arxiv.org/html/2607.05531#bib.bib46)\)formalize collapse as a phase transition, while Ichikawa et al\.Ichikawa and Hukushima \([2024](https://arxiv.org/html/2607.05531#bib.bib46)\)derive a closed\-form collapse threshold as a function of data covariance and𝕂​𝕃\\mathbb\{KL\}weight\.

Alternatively, using the ELBO decompositionHoffman and Johnson \([2016](https://arxiv.org/html/2607.05531#bib.bib27)\), InfoVAEZhaoet al\.\([2019](https://arxiv.org/html/2607.05531#bib.bib40)\)introduces separate weights forIq​\(X;Z\)I\_\{q\}\(X;Z\)and the marginal, enforcing aggregate matching via maximum mean discrepancy \(MMD\)\. Wasserstein autoencodersTolstikhinet al\.\([2017](https://arxiv.org/html/2607.05531#bib.bib33)\)match the aggregate using optimal transport; adversarial approachesMakhzaniet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib36)\); Meschederet al\.\([2017](https://arxiv.org/html/2607.05531#bib.bib37)\)use discriminator training\. All require integration over the dataset to estimateq​\(z\)q\(z\), adding compute cost and training complexity\. VampPriorTomczak and Welling \([2018](https://arxiv.org/html/2607.05531#bib.bib28)\)learns a flexible prior as a mixture of posteriors, reducing structural mismatch but requiring Monte Carlo𝕂​𝕃\\mathbb\{KL\}estimation\. Our duality result Eq\.[8](https://arxiv.org/html/2607.05531#S3.E8)shows thatλ\\lambda\-VAE addresses marginal mismatch implicitly through the information gap, without aggregate estimation\. Furthermore, we can explicity tradeoff genaration\-reconstruction quality usingδ\\deltain Eq[16](https://arxiv.org/html/2607.05531#S4.E16)\.

## 6Experiments

In this section we empirically validateλ\\lambda\-VAE on synthetic and real datasets, demonstrating its effect on posterior collapse, variance equalization, information flow, and generation quality\.

### 6\.1Dataset and Setup

Datasets:We use a combination of synthetic and collected datasets\. The synthetic dataset is drawn from a mixture of Gaussiansp​\(𝐱\)=∑kπk​𝒩​\(𝐱;μk,σk​I\)p\(\\mathbf\{x\}\)=\\sum\_\{k\}\\pi\_\{k\}\\mathcal\{N\}\(\\mathbf\{x\};\\,\\mu\_\{k\},\\sigma\_\{k\}I\), withk=2k\{=\}2andk=4k\{=\}4components, generating two\-dimensional data\. For binary tasks we use Binarized MNISTSalakhutdinov and Murray \([2008](https://arxiv.org/html/2607.05531#bib.bib58)\); Burdaet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib23)\)and Binarized OMNIGLOTLakeet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib57)\); Burdaet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib23)\); pixels are binarized stochastically as described inBurdaet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib23)\)\. For RGB tasks we use CIFAR\-10Krizhevskyet al\.\([2009](https://arxiv.org/html/2607.05531#bib.bib54)\)\(32×3232\{\\times\}32, 10 classes\) and CelebA\-64Liuet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib55)\)\(face images center\-cropped and resized to64×6464\{\\times\}64\)\. All datasets use standard train–test splits and all results are reported on the test set\.

Models:For synthetic and binarized datasets we use a 3\-layer MLP with 300 hidden units per layer for both encoder and decoder\. For RGB datasets we use an*asymmetric*ResNet: a weak encoder \(base channels 16, 1 residual block per stage, 3 downsampling stages, channel multipliers\[1,2,4\]\[1,2,4\]\) paired with a strong decoder \(base channels 64, 2 residual blocks per stage, 3 upsampling stages\)\. The latent dimension isK=2K\{=\}2\(synthetic\),K=30K\{=\}30\(MNIST and OMNIGLOT\),K=512K\{=\}512\(CIFAR\-10\), andK=1024K\{=\}1024\(CelebA\-64\)\. We use Bernoulli likelihood for binarized datasets and discretized logistic mixture \(10 components\) for RGB datasets\. All likelihoods model pixels as statistically independent except in the cases where PixelCNN decoderGulrajaniet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib22)\)is used– we explicitly state it in such a case\.

Training:Binary models are trained for 200 epochs with AdamKingma and Ba \([2014](https://arxiv.org/html/2607.05531#bib.bib49)\)\(lr=0\.001=0\.001\)\. Models on Natural Images are trained for 400 epochs on 8 GPUs using Distributed Data Parallelism \(DDP\) with linear learning rate scaling \(effective lr=8×10−4=8\{\\times\}10^\{\-4\}\)Goyalet al\.\([2017](https://arxiv.org/html/2607.05531#bib.bib44)\), cosine decay to lr=min10−5\{\}\_\{\\text\{min\}\}=10^\{\-5\}, and gradient clipping at 1\.0, with 100 batch size\. For optimalλ∗\\lambda^\{\*\}, the per\-dimension values are updated every 5 epochs using an exponential moving average \(EMA\) ofσ\\sigma\(decay0\.90\.9\) and Eq\.[16](https://arxiv.org/html/2607.05531#S4.E16); a linear ramp of 150 epochs is applied to prevent premature compression before the encoder stabilizes\. Meanwhile, the optimalλ\\lambdain both binary and synthetic dataset is updated every 2 epochs\. We setδ=1\.01\\delta\{=\}1\.01for binary datasets andδ=1\.001\\delta\{=\}1\.001for image datasets, unless we are reporting an ablation study on the hyperparameter\. The baseline is an identical\-architecture VAE with fixedλ=1\\lambda\{=\}1throughout training\. Hence, the online difference between optimalλ\\lambda\-VAE and Standard VAE, is the per\-dimensionλ\\lambdaupdate according to Eq\.[16](https://arxiv.org/html/2607.05531#S4.E16)\.

Metrics:We report NLL \(nats\) and bits per dimensionBPD=NLL/\(D​ln⁡2\)\\text\{BPD\}=\\text\{NLL\}\\,/\\,\(D\\ln 2\)computed as the ELBO on the full test set222IWAEBurdaet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib23)\)is systematically biased forλ\\lambda\-VAE models: whenσλd≪1\\sigma^\{\\lambda\_\{d\}\}\\ll 1all importance samples cluster nearμ\\mu, collapsing the effective sample size and overestimating NLL\. Hence, we report ELBO\-BPD as the primary metric\.\. Active units \(AU\) count dimensions withAzi=Covx​\(𝔼​\[zi\|x\]\)\>0\.01A\_\{z\_\{i\}\}=\\mathrm\{Cov\}\_\{x\}\\\!\\left\(\\mathbb\{E\}\[z\_\{i\}\|x\]\\right\)\>0\.01Burdaet al\.\([2015](https://arxiv.org/html/2607.05531#bib.bib23)\)\. For binary and image datasets we estimateI​\(X;Z\)I\(X;Z\)via the SNR\-based*information capacity*derived directly from the Gaussian channel model of Section[4\.2](https://arxiv.org/html/2607.05531#S4.SS2)\(Eq\.[7](https://arxiv.org/html/2607.05531#S3.E7)\)\. For synthetic data the trueI​\(X;Z\)I\(X;Z\)is bounded by theK=2K\{=\}2latent space and remains well belowlog⁡N\\log Nfor any practicalNN; we therefore use the Monte Carlo mixture marginalisation estimatorI^=H​\(Z\)−H​\(Z\|X\)\\hat\{I\}=H\(Z\)\-H\(Z\|X\), whereH​\(Z\)H\(Z\)is approximated by theNN\-component posterior mixture andH​\(Z\|X\)H\(Z\|X\)is evaluated in closed form\. We reportCCasCapacityin Tables[1](https://arxiv.org/html/2607.05531#S6.T1)and[2](https://arxiv.org/html/2607.05531#S6.T2), and the information gapI​\(X;ϕ\)−CI\(X;\\phi\)\-Cto track how much encoder capacity reaches the decoder\. We additionally reportdecoder capacity∑iCidec\\sum\_\{i\}C\_\{i\}^\{\\mathrm\{dec\}\}, where

Cidec=𝔼x​\[\(∂log⁡p​\(x\|z\)∂zi\)2\]\\displaystyle C\_\{i\}^\{\\mathrm\{dec\}\}\\;=\\;\\sqrt\{\\,\\mathbb\{E\}\_\{x\}\\\!\\left\[\\\!\\left\(\\frac\{\\partial\\log p\(x\|z\)\}\{\\partial z\_\{i\}\}\\right\)^\{\\\!2\}\\right\]\}\(18\)is the RMS decoder gradient per latent dimension \(Tables[2](https://arxiv.org/html/2607.05531#S6.T2)and[3](https://arxiv.org/html/2607.05531#S7.T3)\); a dimension withCidec≈0C\_\{i\}^\{\\mathrm\{dec\}\}\\approx 0contributes nothing to the reconstruction regardless ofσi\\sigma\_\{i\}\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x5.png)\(a\)Gradient imbalance \(k=2k\{=\}2\): standard VAE\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x6.png)\(b\)Gradient imbalance \(k=2k\{=\}2\):λ\\lambda\-VAE \(δ=1\.001\\delta\{=\}1\.001\)\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x7.png)\(c\)Information gap \(k=4k\{=\}4\): standard VAE\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x8.png)\(d\)Information gap \(k=4k\{=\}4\):λ\\lambda\-VAE \(δ=1\.001\\delta\{=\}1\.001\)\.

Figure 3:Gradient imbalance and information gap on synthetic Gaussian mixtures\.Each panel tracks the gradient ratioρi\\rho\_\{i\}and mutual informationI​\(X;Z\)I\(X;Z\)over training, alongside the latent space\.*Top row*\(k=2k\{=\}2, clusters stretched along one axis\), isolating gradient imbalance\. In the standard VAE the reconstruction gradientgiσg\_\{i\}^\{\\sigma\}collapses to zero for the low\-variance dimension beforeσi→1\\sigma\_\{i\}\\to 1, confirming Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1); the latent space shows complete information loss as well\.λ\\lambda\-VAE amplifies the gradient byλ​σiλ−1\\lambda\\sigma\_\{i\}^\{\\lambda\-1\}, keepingρi\\rho\_\{i\}elevated and the dimension active throughout training\.*Bottom row*\(k=4k\{=\}4, equal inter\-cluster separation along each axis\): the standard VAE exhibits a fluctuating non\-zero information gap, resulting in variation loss in both the latent and reconstruction spaces\. Underλ\\lambda\-VAE withλ∗\>2\\lambda^\{\*\}\{\>\}2for all dimensions the gap is driven to zero and the four\-cluster structure is recovered, consistent with Section[4\.2](https://arxiv.org/html/2607.05531#S4.SS2)\.Table 1:Comparison on binarized benchmarks \(MNIST and Omniglot,K=30K\{=\}30\)\.λ\\lambda\-VAE and the ablations shown \(β\\beta\-VAE, standard VAE\) use a three\-layer MLP encoder–decoder; most published baselines use deeper convolutional architectures, making these comparisons indicative rather than directly controlled\. VampPrior is an exception, using a comparable MLP architecture\. NLL and BPD measure reconstruction quality; AU counts active latent dimensions; Capacity is the SNR information capacity, measuring how much information reaches the decoder\. Results for external baselines are taken from original papers\.![Refer to caption](https://arxiv.org/html/2607.05531v1/x9.png)\(a\)Standard VAE:σ\\sigmahistogram and trajectory\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x10.png)\(b\)β\\beta\-VAE \(β=0\.5\\beta\{=\}0\.5\):σ\\sigmahistogram and trajectory\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x11.png)\(c\)λ\\lambda\-VAE \(δ=1\.001\\delta=1\.001\): variance equalization\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x12.png)\(d\)λ¯∗\\bar\{\\lambda\}^\{\*\}trajectory \(mean±\\pmstd over dimensions\)

Figure 4:Variance equalisation on Binary\-MNIST \(K=30K\{=\}30\)\.Panels \(a\)–\(c\) each show the per\-dimensionσ\\sigmahistogram alongside the mean±\\pmstd trajectory over training epochs\.*\(a\)*Standard VAE: the distribution becomes bimodal; collapsed dimensions accumulate nearσ=1\\sigma\{=\}1while active dimensions cluster at lower values\.*\(b\)*β\\beta\-VAE \(β=0\.5\\beta\{=\}0\.5\): the mode shifts downward but a tail nearσ=1\\sigma\{=\}1persists\.*\(c\)*λ\\lambda\-VAE \(δ=1\.001\\delta\{=\}1\.001\): all dimensions converge to a tight band aroundσ∗\\sigma^\{\*\}; the across\-dimension std collapses to near zero, realising variance equalization \(Section[4\.3](https://arxiv.org/html/2607.05531#S4.SS3)\)\.*\(d\)*Correspondingλ¯\\bar\{\\lambda\}trajectory \(mean±\\pmstd over dimensions\):λ¯\\bar\{\\lambda\}stabilises at≈5\.3\{\\approx\}\\,5\.3, confirming jointσ∗\\sigma^\{\*\}–λ∗\\lambda^\{\*\}convergence\.
### 6\.2Collapse Prevention, Information Recovery, and Reconstruction Quality

We validateλ\\lambda\-VAE along three axes, suppression of gradient imbalance, reduction of the information gap, and downstream reconstruction quality\.

Synthetic data:We ran a standard VAE andλ\\lambda\-VAE on two synthetickk\-component Gaussian mixtures withK=2K\{=\}2latent dimensions andδ=1\.001\\delta\{=\}1\.001, designed to simulate the two causes of collapse independently \(Figure[3](https://arxiv.org/html/2607.05531#S6.F3)\)\. Fork=2k\{=\}2, the reconstruction gradient norm of the standard VAE’s first dimension collapses to zero beforeσ1→1\\sigma\_\{1\}\\to 1, matching Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1); the latent space and reconstructions confirm complete information loss from that dimension\.λ\\lambda\-VAE amplifies the gradient byλ​σλ−1\\lambda\\sigma^\{\\lambda\-1\}, keeping the dimension active throughout training; compare Figure[3\(a\)](https://arxiv.org/html/2607.05531#S6.F3.sf1)vs[3\(b\)](https://arxiv.org/html/2607.05531#S6.F3.sf2)\. Fork=4k\{=\}4, the standard VAE exhibits a fluctuating yet non\-zero information gap that suppresses variation in both latent and data clusters, while introducing high reconstruction error per dimension leading to noisy data recovery\. Underλ\\lambda\-VAE, however, sinceλ∗\>2\\lambda^\{\*\}\>2for all dimensions the gap is driven to zero, consistent with Section[4\.2](https://arxiv.org/html/2607.05531#S4.SS2), while both latent and reconstruction spaces preserve the four\-cluster structure relatively well\.

Binary data:On Binarized MNIST and Binarized OMNIGLOT \(K=30K\{=\}30, three\-layer MLP\), the standard VAE collapses 16 of 30 dimensions on MNIST \(AU=14\\,\{=\}\\,14\) and 13 of 30 on OMNIGLOT \(AU=17\\,\{=\}\\,17\)\.λ\\lambda\-VAE \(δ=1\.01\\delta\{=\}1\.01\) reduces this to one on MNIST \(AU=29/30\\,\{=\}\\,29/30\) and zero on OMNIGLOT \(AU=30/30\\,\{=\}\\,30/30\), as reported in Table[1](https://arxiv.org/html/2607.05531#S6.T1)\. In Figure[4](https://arxiv.org/html/2607.05531#S6.F4), the mechanism is illustrated viaσ\\sigmadistributions\. The standard VAE is bimodal with a mass nearσ=1\\sigma\{=\}1, whileβ\\beta\-VAE shifts the distribution but does not eliminate the tail\. Theλ\\lambda\-VAE, however, concentrates all mass in a narrow band aroundσ≈0\.4\\sigma\\approx 0\.4, realizing the variance equalization of Section[4\.3](https://arxiv.org/html/2607.05531#S4.SS3)\. The average per\-dimension estimate stabilizes atλ¯≈5\.3\\bar\{\\lambda\}\\approx 5\.3on MNIST andλ¯≈2\.5\\bar\{\\lambda\}\\approx 2\.5on OMNIGLOT atδ=1\.01\\delta\{=\}1\.01, reflecting their different active\-dimensionσ\\sigmadistributions; Section[6\.3](https://arxiv.org/html/2607.05531#S6.SS3)confirms the theoretical predictive curve Eq\.[16](https://arxiv.org/html/2607.05531#S4.E16)aligns with empirical data\.

The Capacity column in Table[1](https://arxiv.org/html/2607.05531#S6.T1)separates two mechanisms for increasing information throughput, computed as Eq\.[7](https://arxiv.org/html/2607.05531#S3.E7)\.β\\beta\-VAE \(β=0\.5\\beta\{=\}0\.5\) achieves comparable capacity toλ\\lambda\-VAE \(δ=1\.1\\delta\{=\}1\.1\) on both datasets \(34\.2 vs\. 37\.2 nats on MNIST; 38\.4 vs\. 37\.4 on OMNIGLOT\), but does so by loosening the𝕂​𝕃\\mathbb\{KL\}coefficient, inflatingVarx​\[μd\]\\mathrm\{Var\}\_\{x\}\[\\mu\_\{d\}\]across active dimensions at the expense of prior matching\. The consequence is substantially worse BPD \(0\.242 vs\. 0\.203 on OMNIGLOT\), while the aggregate posterior drifts from𝒩​\(0,𝐈\)\\mathcal\{N\}\(0,\\mathbf\{I\}\)\. Provided a calibratedδ\\deltais chosen,λ\\lambda\-VAE instead reduces the effective noiseσdλd\\sigma\_\{d\}^\{\\lambda\_\{d\}\}per dimension without weakening the prior\. Atδ=1\.01\\delta\{=\}1\.01, stronger per\-dimension compression \(avg\.λ¯≈5\.3\\bar\{\\lambda\}\{\\approx\}5\.3on MNIST\) drives Capacity to 69\.4 nats \(nearly3×3\\timesthe standard VAE\), while simultaneously achieving the best BPD\. Meanwhile,δ=1\.19\\delta=1\.19achieves a comparable AU, despite a significantly lower information throughput – see Section[6\.3](https://arxiv.org/html/2607.05531#S6.SS3)for discussion on effects ofδ\\delta\.

Natural images:On RGB images, the fixed\-λ\\lambdamodel on CIFAR\-10 leaves 173 of 512 dimensions \(AU=339\), i\.e\., dimensions where the optimal schedule would assignλd∗≫1\\lambda^\{\*\}\_\{d\}\\gg 1but fixedλ=1\\lambda\{=\}1provides no correction\. The SNR information capacity Eq\.[7](https://arxiv.org/html/2607.05531#S3.E7)\(Table[2](https://arxiv.org/html/2607.05531#S6.T2)\) sits at 628 nats\. On the contrary under optimalλ\\lambda, all 512 dimensions are driven belowσ=0\.9\\sigma\{=\}0\.9and the effective noiseσdλd\\sigma\_\{d\}^\{\\lambda\_\{d\}\}is equalized across the full latent code;CSNRC\_\{\\mathrm\{SNR\}\}grows to 1760 nats \(2\.8×2\.8\\timesimprovement\), as shown in Figure[6\(a\)](https://arxiv.org/html/2607.05531#S6.F6.sf1)\. Decoder capacity rises in parallel from 169 to 361 nats \(2\.1×2\.1\\times, Figure[6\(b\)](https://arxiv.org/html/2607.05531#S6.F6.sf2)\) as a consequence of gradient rebalancing\.

On CelebA\-64 \(K=1024K\{=\}1024\),CSNRC\_\{\\mathrm\{SNR\}\}grows from 2198 to 3646 nats \(1\.7×1\.7\\times\) and decoder capacity from 3283 to 6596 nats \(2\.0×2\.0\\times\)\. These gains translate directly into reconstruction quality\. ELBO\-BPD improves by\+0\.33\+0\.33on CIFAR\-10 \(7\.1% relative\) and\+0\.24\+0\.24on CelebA\-64 \(6\.2% relative\), see Table[2](https://arxiv.org/html/2607.05531#S6.T2)\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x13.png)\(a\)Reconstruction vs Random sampling
![Refer to caption](https://arxiv.org/html/2607.05531v1/x14.png)\(b\)Prior samples ELBO

Figure 5:Reconstruction vs Random sampling\.Reconstruction and prior sampling on Omniglot; figures best viewed zoomed\. \(a\) Top: test reconstructions \(z=μz\{=\}\\mu\); bottom: samples fromz∼𝒩​\(0,I\)z\\sim\\mathcal\{N\}\(0,I\)displayed as decoder probabilities as pixel intensity for prob \> 0\.5 \(greedy sampling of Bernoulli\)\. Random samples are shared across models for the decoding\. \(b\) ELBO on 256 prior samples per model variant\. The figures show thatδ=1\.19\\delta\{=\}1\.19retains reconstruction fidelity while producing coherent prior samples;δ=1\.01\\delta\{=\}1\.01achieves sharp reconstructions but loses generative quality as the aggregate posterior drifts from𝒩​\(0,I\)\\mathcal\{N\}\(0,I\), confirming thatδ\\deltacan be calibrated to close the information gap without over\-compressing the latent code\.
### 6\.3Optimalλ\\lambdaand Its Sensitivity toδ\\delta

λ∗\\lambda^\{\*\}estimate alignment with theory:Figure[7](https://arxiv.org/html/2607.05531#S6.F7)plots the per\-dimension\(σ,λ∗\)\(\\sigma,\\lambda^\{\*\}\)scatter for all latent dimensions in CIFAR\-10 and CelebA\-64 at epoch 400\. Every dimension lies exactly on the theoretical curve defined in Proposition[3](https://arxiv.org/html/2607.05531#Thmprop3), confirming that the EMA\-based update converges to the analytic optimum derived in Appendix[Appendix B](https://arxiv.org/html/2607.05531#Ax1)\. The fixed model’s dimensions form a horizontal line atλ=1\\lambda\{=\}1across the fullσ\\sigmarange; the 173 dimensions withσ\>0\.9\\sigma\>0\.9would receiveλ∗≫1\\lambda^\{\*\}\\gg 1under the optimal schedule, representing a systematic under\-utilisation that fixedλ\\lambdacannot correct\. The same alignment holds for CelebA\-64 \(K=1024K\{=\}1024;λ¯=1\.99\\bar\{\\lambda\}\{=\}1\.99,λmax=10\.4\\lambda\_\{\\max\}\{=\}10\.4at convergence\)\. The convergence ofλ∗\\lambda^\{\*\}andσ∗\\sigma^\{\*\}is also shown in Figure[4](https://arxiv.org/html/2607.05531#S6.F4)for Binary MNIST dataset\.

Effect ofδ\\delta:The parameterδ\\deltadrives the equilibriumσ∗\\sigma^\{\*\}via optimal estimate Eq\.[16](https://arxiv.org/html/2607.05531#S4.E16)\. It controls the aggressiveness of compression \(how close samplezzis toμ\\mu\)\. Smallerδ\\deltainduces largerλ∗\\lambda^\{\*\}, compressing more dimensions more tightly\. Table[1](https://arxiv.org/html/2607.05531#S6.T1)illustrates this particularly on binary MNIST dataset; reducingδ\\deltafrom 1\.19 to 1\.01 increases AU from 21/30 to 29/30 and improves ELBO\-BPD from 0\.172 to 0\.144\. The AU ceiling, however, is reached atδ=1\.1\\delta\{=\}1\.1for OMNIGLOT\.

As discussed in Section[4\.4](https://arxiv.org/html/2607.05531#S4.SS4), the tradeoff here is increased marginal divergence for improved information flow, which consequently affects the quality of sampling from a prior\. This can be calibrated as required\. Figure[5](https://arxiv.org/html/2607.05531#S6.F5)shows thatδ=1\.19\\delta\{=\}1\.19navigates this tradeoff effectively – it improves reconstruction, BPD 0\.204 vs\. 0\.226 for the standard VAE on OMNIGLOT, while achieving*better*prior sample quality than all compared methods \(Standard VAE,β\\beta\-VAE and differentδ\\deltavalues\) with−1776\-1776ELBO, estimated on 256 prior samples\. Meanwhile,δ=1\.01\\delta\{=\}1\.01, while minimising BPD \(0\.189\), substantially degrades generative quality \(−2167\-2167ELBO\)\. Consequently, choosing the rightδ\\deltaleads to maximized net information gain Eq\.[15](https://arxiv.org/html/2607.05531#S4.E15), with little or no compromise on the generative quality\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x15.png)\(a\)SNR information capacityCSNRC\_\{\\mathrm\{SNR\}\}
![Refer to caption](https://arxiv.org/html/2607.05531v1/x16.png)\(b\)Decoder capacity\.

Figure 6:SNR information capacity and decoder capacity over training\.\(a\) Under optimalλ\\lambda,CSNRC\_\{\\mathrm\{SNR\}\}grows from 628 to 1760 nats on CIFAR\-10 \(2\.8×2\.8\\times\) and from 2198 to 3646 nats on CelebA\-64 \(1\.7×1\.7\\times\)\. \(b\) Decoder capacity rises in parallel: 169→\\to361 nats \(2\.1×2\.1\\times\) on CIFAR\-10 and 3283→\\to6596 nats \(2\.0×2\.0\\times\) on CelebA\-64, confirming that the increased encoder throughput is actively utilised by the decoder\.Table 2:Controlled comparison on CIFAR\-10 and CelebA\-64\.Fixed and optimalλ\\lambdashare the same asymmetric ResNet architecture\. SNR capacity information throughput; decoder capacity \(Figure[6\(b\)](https://arxiv.org/html/2607.05531#S6.F6.sf2)\) measures the information each latent dimension actively contributes to the decoder output\.![Refer to caption](https://arxiv.org/html/2607.05531v1/x17.png)Figure 7:Empirical alignment with the optimalλ∗\\lambda^\{\*\}schedule\.All 512 CIFAR\-10 and 1024 CelebA\-64 dimensions lie exactly on the theoretical curve Eq\.[16](https://arxiv.org/html/2607.05531#S4.E16)at convergence \(Pearsonr=1\.00r=1\.00\)\. Points nearσ≈0\\sigma\\approx 0haveλ∗→λmax=20\\lambda^\{\*\}\\to\\lambda\_\{\\max\}=20\(clamped\); points nearσ=1\\sigma=1are dimensions the vanilla VAE model leaves dead\.
### 6\.4Qualitative Comparison

In this section, we show the impact of increased information flow and minimized posterior collapse on reconstruction quality\.

In Figure[8](https://arxiv.org/html/2607.05531#S6.F8), we compare reconstructions between the standard and optimal\-λ\\lambdaVAE\.λ\\lambda\-VAE produces sharper images with more coherent pixel values on both CIFAR\-10 and CelebA\-64\. This qualitative improvement follows directly from the2\.8×2\.8\\timesgain in SNR information capacity and the2\.1×2\.1\\timesgain in decoder capacity shown in Figure[6](https://arxiv.org/html/2607.05531#S6.F6)and Table[2](https://arxiv.org/html/2607.05531#S6.T2)\. Reconstruction quality degradation due to posterior collapse, however, is much more prominently observed in a case where the modelling is primarily absorbed by the decoder\. A sufficiently powerful decoder trained with standard VAE causes significant gradient imbalance that can render most of the latent code uninformative and wasted\. This phenomenon is more apparent in autoregressive models like pixelCNN decodersGulrajaniet al\.\([2016](https://arxiv.org/html/2607.05531#bib.bib22)\)where the decoder is trained with teacher forcing and can quickly learn to ignore most of the latent codes, especially pixel level information when it is conditioned on the ground\-truth of previous pixel\. Hence, leading to a significant posterior collapse and poor sample quality during*inference*, despite a comparably good BPD during training\. In contrast, the optimalλ\\lambdakeepsziz\_\{i\}close toμi\\mu\_\{i\}with high SNR throughout training with reasonable variance \(depending onδ\\delta\), ensuring the decoder cannot bypass the latent code regardless of decoder capacity\. We show this contrast by using the same architecture as before only replacing the decoder with a PixelCNN decoder\. Both models achieve comparable ELBO\-BPD at evaluation \(3\.518 standard vs\. 3\.494 optimal; Table[3](https://arxiv.org/html/2607.05531#S7.T3)\), yet 378 of 512 latent dimensions are effectively collapsed in the standard VAE versus none in the optimalλ\\lambdamodel\. Decoder capacity further confirms this; the standard VAE’s PixelCNN allocates only 12\.1 nats to the latent code whileλ\\lambda\-VAE allocates 75\.0 nats \(6\.2×6\.2\{\\times\}more\), meaning the autoregressive decoder has learned to model CIFAR\-10 almost entirely from its own context\. The collapse is exposed when sampling from the model in both*greedy*mode \(argmax at each pixel conditioned onzzand all previous pixels\) and*ancestral*sampling \(p​\(xi\|x<i,z\)p\(x\_\{i\}\|x\_\{<i\},z\)drawn sequentially per pixel\), as shown in Figure[9](https://arxiv.org/html/2607.05531#S7.F9)\. This is primarily due to preserved information flow under optimalλ\\lambda\-VAE even though teacher forcing was used in both cases for training\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x18.png)\(a\)CelebA\-64
![Refer to caption](https://arxiv.org/html/2607.05531v1/x19.png)\(b\)CIFAR10

Figure 8:Qualitative Comparison On Reconstructions\. Optimalλ\\lambdaproduces sharper, more detailed reconstructions, consistent with the 2\.1× capacity recovery in Table[2](https://arxiv.org/html/2607.05531#S6.T2)\. Differences in quality are best observed zoomed in\.

## 7Conclusion

Posterior collapse is commonly attributed to the𝕂​𝕃\\mathbb\{KL\}term overwhelming reconstruction, but this framing misses the mechanism\. The𝕂​𝕃\\mathbb\{KL\}term does not suppress all dimensions uniformly – it drives*specific*dimensions towardσi=1\\sigma\_\{i\}=1through two coupled pathways\. Atσi=1\\sigma\_\{i\}=1, both pathways converge: it is simultaneously a stable fixed point of the training dynamics where the reconstruction gradient vanishes \(Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1)\), and the regime in which injected noise overwhelms the encoded signal, reducing per\-channel SNR and decoupling the decoder from the latent code \(Proposition[2](https://arxiv.org/html/2607.05531#Thmprop2)\)\.λ\\lambda\-VAE corrects both by shifting the stable attractor fromσi=1\\sigma\_\{i\}=1toσ∗=λ−1/\(λ−1\)<1\\sigma^\{\*\}=\\lambda^\{\-1/\(\\lambda\-1\)\}<1, simultaneously restoring the reconstruction gradient and calibrating per\-channel noise to signal strength\. The per\-dimension optimalλ\\lambdaestimation \(Proposition[3](https://arxiv.org/html/2607.05531#Thmprop3)\) follows from a net information gain objective that balances information throughput against marginal\-prior divergence, withδ\\deltaas the only hyperparameter\. Empirically,λ\\lambda\-VAE consistently reduces collapsed dimensions, recovers information throughput, and improves reconstruction quality across all tested scales – synthetic, binarized, and large\-scale RGB – with the PixelCNN experiment providing the sharpest demonstration with near\-identical BPD \(3\.518 vs\. 3\.494\) yet6\.2×6\.2\\timesmore decoder capacity allocated to the latent code\.

The broader implications for general training principles can be summarized as follows: a stochastic bottleneck trained end\-to\-end with a sufficiently powerful downstream module will be bypassed when the injected noise exceeds the signal capacity\. This is not because the model cannot learn, but because bypassing is the lower\-cost solution under a uniform noise budget\.λ\\lambda\-VAE instantiates a targeted correction for Gaussian posteriors, but the same principle is expected to apply wherever the noise level at a stochastic layer is set globally rather than calibrated to per\-channel signal strength\. Three directions are left for future work\.*Hierarchical VAEs:*collapse at early latent levels propagates through the hierarchy; per\-levelλ∗\\lambda^\{\*\}schedules may provide targeted repair without the careful𝕂​𝕃\\mathbb\{KL\}balancing that hierarchical training currently requires\.*Adaptiveδ\\delta:*the current approach setsδ\\deltaonce per experiment; learningδ\\deltaor scheduling it based on the evolvingσ\\sigmadistribution could recover the reconstruction, generation tradeoff automatically, and reduce per\-dataset tuning\.*Beyond VAEs:*the information gap–marginal mismatch duality \(Eq\.[8](https://arxiv.org/html/2607.05531#S3.E8)\) provides a cheap, dataset\-free proxy for aggregate posterior quality that avoids the integration cost of InfoVAE or Wasserstein objectives; whether this proxy can guide training in discrete bottleneck models or diffusion\-based posteriors is an open question\.

![Refer to caption](https://arxiv.org/html/2607.05531v1/x20.png)\(a\)Greedy decoding \(argmax at each pixel step\)\.
![Refer to caption](https://arxiv.org/html/2607.05531v1/x21.png)\(b\)Ancestral sampling fromp​\(xi\|x<i,z\)p\(x\_\{i\}\|x\_\{<i\},z\)\.

Figure 9:Reconstruction with PixelCNN decoder on CIFAR\-10\.Each panel shows ground\-truth \(left\), Standard VAE \(middle\), andλ\\lambda\-VAE \(right\) for the same test images\. Despite near\-identical ELBO\-BPD \(3\.518 vs\. 3\.494, Table[3](https://arxiv.org/html/2607.05531#S7.T3)\), the standard VAE has 378/512 collapsed latent dimensions and decoder capacity of only 12\.1 nats: the PixelCNN has learned to model the image from pixel context alone and ignoreszz\. This collapse is visible in both sampling modes \(greedy and ancestral\) whereλ\\lambda\-VAE reconstructions retain input\-specific detail even without teacher forcing\.Table 3:PixelCNN decoder comparison on CIFAR\-10\.BPD evaluated via the test\-set ELBO\. Decoder capacity sums per\-dimension contributions from the training history \(a proxy for how much the decoder relies onzz\)\.
## References

- Y\. Bengio, A\. Courville, and P\. Vincent \(2013\)Representation learning: a review and new perspectives\.IEEE transactions on pattern analysis and machine intelligence35\(8\),pp\. 1798–1828\.Cited by:[§3\.2](https://arxiv.org/html/2607.05531#S3.SS2.p4.5),[§4\.3](https://arxiv.org/html/2607.05531#S4.SS3.p2.1)\.
- S\. R\. Bowman, L\. Vilnis, O\. Vinyals, A\. M\. Dai, R\. Jozefowicz, and S\. Bengio \(2015\)Generating sentences from a continuous space\.arXiv preprint arXiv:1511\.06349\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1),[§3\.1](https://arxiv.org/html/2607.05531#S3.SS1.p3.12)\.
- Y\. Burda, R\. Grosse, and R\. Salakhutdinov \(2015\)Importance weighted autoencoders\.arXiv preprint arXiv:1509\.00519\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p1.5),[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p4.14),[footnote 2](https://arxiv.org/html/2607.05531#footnote2)\.
- C\. P\. Burgess, I\. Higgins, A\. Pal, L\. Matthey, N\. Watters, G\. Desjardins, and A\. Lerchner \(2018\)Understanding disentangling in beta\-vae\.arXiv preprint arXiv:1804\.03599\.Cited by:[§5](https://arxiv.org/html/2607.05531#S5.p2.12)\.
- X\. Chen, D\. P\. Kingma, T\. Salimans, Y\. Duan, P\. Dhariwal, J\. Schulman, I\. Sutskever, and P\. Abbeel \(2016\)Variational lossy autoencoder\.arXiv preprint arXiv:1611\.02731\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1),[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p2.12)\.
- T\. M\. Cover \(1999\)Elements of information theory\.John Wiley & Sons\.Cited by:[Appendix A](https://arxiv.org/html/2607.05531#A1.SSx2.p1.1),[§3\.2](https://arxiv.org/html/2607.05531#S3.SS2.p1.3)\.
- H\. Fu, C\. Li, X\. Liu, J\. Gao, A\. Celikyilmaz, and L\. Carin \(2019\)Cyclical annealing schedule: a simple approach to mitigating kl vanishing\.arXiv preprint arXiv:1903\.10145\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p2.12)\.
- P\. Goyal, P\. Dollár, R\. Girshick, P\. Noordhuis, L\. Wesolowski, A\. Kyrola, A\. Tulloch, Y\. Jia, and K\. He \(2017\)Accurate, large minibatch sgd: training imagenet in 1 hour\.arXiv preprint arXiv:1706\.02677\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p3.12)\.
- I\. Gulrajani, K\. Kumar, F\. Ahmed, A\. A\. Taiga, F\. Visin, D\. Vazquez, and A\. Courville \(2016\)Pixelvae: a latent variable model for natural images\.arXiv preprint arXiv:1611\.05013\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p2.5),[§6\.4](https://arxiv.org/html/2607.05531#S6.SS4.p2.14)\.
- J\. He, D\. Spokoyny, G\. Neubig, and T\. Berg\-Kirkpatrick \(2019\)Lagging inference networks and posterior collapse in variational autoencoders\.arXiv preprint arXiv:1901\.05534\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p4.1)\.
- I\. Higgins, L\. Matthey, A\. Pal, C\. Burgess, X\. Glorot, M\. Botvinick, S\. Mohamed, and A\. Lerchner \(2016\)Beta\-vae: learning basic visual concepts with a constrained variational framework\.InInternational conference on learning representations,Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p2.12),[Table 1](https://arxiv.org/html/2607.05531#S6.T1.12.6.2)\.
- G\. E\. Hinton \(1986\)Learning distributed representations of concepts\.InProceedings of the Annual Meeting of the Cognitive Science Society,Vol\.8\.Cited by:[§3\.2](https://arxiv.org/html/2607.05531#S3.SS2.p4.5),[§4\.3](https://arxiv.org/html/2607.05531#S4.SS3.p2.1)\.
- M\. D\. Hoffman and M\. J\. Johnson \(2016\)Elbo surgery: yet another way to carve up the variational evidence lower bound\.InWorkshop in Advances in Approximate Bayesian Inference, NIPS,Vol\.1\.Cited by:[Appendix A](https://arxiv.org/html/2607.05531#A1.SSx2.SSS0.Px3.p1.1),[§2](https://arxiv.org/html/2607.05531#S2.p1.8),[§3\.2](https://arxiv.org/html/2607.05531#S3.SS2.p5.1),[§5](https://arxiv.org/html/2607.05531#S5.p5.5)\.
- Y\. Ichikawa and K\. Hukushima \(2024\)Learning dynamics in linear vae: posterior collapse threshold, superfluous latent space pitfalls, and speedup with kl annealing\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 1936–1944\.Cited by:[§5](https://arxiv.org/html/2607.05531#S5.p4.1)\.
- M\. I\. Jordan, Z\. Ghahramani, T\. S\. Jaakkola, and L\. K\. Saul \(1999\)An introduction to variational methods for graphical models\.Machine learning37,pp\. 183–233\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- D\. P\. Kingma and J\. Ba \(2014\)Adam: a method for stochastic optimization\.arXiv preprint arXiv:1412\.6980\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p3.12)\.
- D\. P\. Kingma, D\. J\. Rezende, S\. Mohamed, and M\. Welling \(2014\)Semi\-supervised learning with deep generative models\.Advances in neural information processing systems27\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- D\. P\. Kingma and M\. Welling \(2013\)Auto\-encoding variational bayes\.arXiv preprint arXiv:1312\.6114\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1),[§2](https://arxiv.org/html/2607.05531#S2.p1.16)\.
- D\. P\. Kingma, T\. Salimans, R\. Jozefowicz, X\. Chen, I\. Sutskever, and M\. Welling \(2016\)Improved variational inference with inverse autoregressive flow\.Advances in neural information processing systems29\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p3.1),[Table 1](https://arxiv.org/html/2607.05531#S6.T1.32.27.1.2)\.
- A\. Krizhevsky, G\. Hinton,et al\.\(2009\)Learning multiple layers of features from tiny images\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p1.5)\.
- B\. M\. Lake, R\. Salakhutdinov, and J\. B\. Tenenbaum \(2015\)Human\-level concept learning through probabilistic program induction\.Science350\(6266\),pp\. 1332–1338\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p1.5)\.
- Z\. Liu, P\. Luo, X\. Wang, and X\. Tang \(2015\)Deep learning face attributes in the wild\.InProceedings of the IEEE international conference on computer vision,pp\. 3730–3738\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p1.5)\.
- J\. Lucas, G\. Tucker, R\. B\. Grosse, and M\. Norouzi \(2019\)Don’t blame the elbo\! a linear vae perspective on posterior collapse\.Advances in Neural Information Processing Systems32\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1),[§5](https://arxiv.org/html/2607.05531#S5.p4.1)\.
- A\. Makhzani, J\. Shlens, N\. Jaitly, I\. Goodfellow, and B\. Frey \(2015\)Adversarial autoencoders\.arXiv preprint arXiv:1511\.05644\.Cited by:[§5](https://arxiv.org/html/2607.05531#S5.p5.5)\.
- G\. J\. McLachlan and T\. Krishnan \(2007\)The em algorithm and extensions\.John Wiley & Sons\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- L\. Mescheder, S\. Nowozin, and A\. Geiger \(2017\)Adversarial variational bayes: unifying variational autoencoders and generative adversarial networks\.InInternational conference on machine learning,pp\. 2391–2400\.Cited by:[§5](https://arxiv.org/html/2607.05531#S5.p5.5),[Table 1](https://arxiv.org/html/2607.05531#S6.T1.32.28.2.1)\.
- T\. P\. Minka \(2013\)Expectation propagation for approximate bayesian inference\.arXiv preprint arXiv:1301\.2294\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- R\. M\. Neal and G\. E\. Hinton \(1998\)A view of the em algorithm that justifies incremental, sparse, and other variants\.InLearning in graphical models,pp\. 355–368\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- A\. Razavi, A\. v\. d\. Oord, B\. Poole, and O\. Vinyals \(2019a\)Preventing posterior collapse with delta\-vaes\.arXiv preprint arXiv:1901\.03416\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p2.12)\.
- A\. Razavi, A\. Van den Oord, and O\. Vinyals \(2019b\)Generating diverse high\-fidelity images with vq\-vae\-2\.Advances in neural information processing systems32\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- D\. J\. Rezende, S\. Mohamed, and D\. Wierstra \(2014\)Stochastic backpropagation and approximate inference in deep generative models\.InInternational conference on machine learning,pp\. 1278–1286\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1),[§2](https://arxiv.org/html/2607.05531#S2.p1.16)\.
- D\. Rezende and S\. Mohamed \(2015\)Variational inference with normalizing flows\.InInternational conference on machine learning,pp\. 1530–1538\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p3.1)\.
- M\. Rosca, B\. Lakshminarayanan, and S\. Mohamed \(2018\)Distribution matching in variational inference\.arXiv preprint arXiv:1802\.06847\.Cited by:[§5](https://arxiv.org/html/2607.05531#S5.p3.1)\.
- R\. Salakhutdinov and I\. Murray \(2008\)On the quantitative analysis of deep belief networks\.InProceedings of the 25th international conference on Machine learning,pp\. 872–879\.Cited by:[§6\.1](https://arxiv.org/html/2607.05531#S6.SS1.p1.5)\.
- Y\. Shi, B\. Paige, P\. Torr,et al\.\(2019\)Variational mixture\-of\-experts autoencoders for multi\-modal deep generative models\.Advances in neural information processing systems32\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p1.1)\.
- N\. Tishby, F\. C\. Pereira, and W\. Bialek \(2000\)The information bottleneck method\.arXiv preprint physics/0004057\.Cited by:[§2](https://arxiv.org/html/2607.05531#S2.p2.1)\.
- I\. Tolstikhin, O\. Bousquet, S\. Gelly, and B\. Schoelkopf \(2017\)Wasserstein auto\-encoders\.arXiv preprint arXiv:1711\.01558\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p5.5)\.
- J\. Tomczak and M\. Welling \(2018\)VAE with a vampprior\.InInternational Conference on Artificial Intelligence and Statistics,pp\. 1214–1223\.Cited by:[§5](https://arxiv.org/html/2607.05531#S5.p5.5),[Table 1](https://arxiv.org/html/2607.05531#S6.T1.32.29.3.1)\.
- S\. Zhao, J\. Song, and S\. Ermon \(2019\)Infovae: balancing learning and inference in variational autoencoders\.InProceedings of the aaai conference on artificial intelligence,Vol\.33,pp\. 5885–5892\.Cited by:[§1](https://arxiv.org/html/2607.05531#S1.p2.2),[§5](https://arxiv.org/html/2607.05531#S5.p5.5),[Table 1](https://arxiv.org/html/2607.05531#S6.T1.32.30.4.1)\.

## Appendix AAppendix

### A\.1 Gradient Imbalance \(Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1)\)

The reconstruction gradient w\.r\.t\.σi\\sigma\_\{i\}is defined as

giσ:=∂R∂σi=𝔼p​\(𝐱\),ϵi​\[∇zilog⁡pθ​\(𝐱\|𝐳\)⋅ϵi\],g\_\{i\}^\{\\sigma\}\\;:=\\;\\frac\{\\partial R\}\{\\partial\\sigma\_\{i\}\}\\;=\\;\\mathbb\{E\}\_\{p\(\\mathbf\{x\}\),\\,\\epsilon\_\{i\}\}\\\!\\left\[\\nabla\_\{z\_\{i\}\}\\log p\_\{\\theta\}\(\\mathbf\{x\}\|\\mathbf\{z\}\)\\cdot\\epsilon\_\{i\}\\right\],\(19\)while the𝕂​𝕃\\mathbb\{KL\}gradient is∂𝕂​𝕃i/∂σi=σi−1/σi\\partial\\mathbb\{KL\}\_\{i\}/\\partial\\sigma\_\{i\}=\\sigma\_\{i\}\-1/\\sigma\_\{i\}\. Combining, the ELBO gradient is

∂ℒ∂σi=giσ\+1σi−σi\.\\frac\{\\partial\\mathcal\{L\}\}\{\\partial\\sigma\_\{i\}\}\\;=\\;g\_\{i\}^\{\\sigma\}\+\\frac\{1\}\{\\sigma\_\{i\}\}\-\\sigma\_\{i\}\.\(A\.1\)From the above, we see the𝕂​𝕃\\mathbb\{KL\}restoring force1/σi−σi1/\\sigma\_\{i\}\-\\sigma\_\{i\}is positive forσi<1\\sigma\_\{i\}<1, and negative forσi\>1\\sigma\_\{i\}\>1\. It drivesσi→1\\sigma\_\{i\}\\to 1whenever it dominates the reconstruction gradientgiσg\_\{i\}^\{\\sigma\}\.

#### Gradient ratio and fixed points:

Setting the ELBO gradient Eq\.[A\.1](https://arxiv.org/html/2607.05531#A1.Ex1)to zero gives the following equilibrium condition:

σi∗=−giσ\+\(giσ\)2\+42∈\(0,1\),\\sigma\_\{i\}^\{\*\}\\;=\\;\\frac\{\-g\_\{i\}^\{\\sigma\}\+\\sqrt\{\(g\_\{i\}^\{\\sigma\}\)^\{2\}\+4\}\}\{2\}\\;\\in\\;\(0,1\),\(A\.2\)
Whengiσ=0g\_\{i\}^\{\\sigma\}=0, this gives the unique positive rootσi∗=1\\sigma\_\{i\}^\{\*\}=1\. Linearising aroundσi=1\+δ\\sigma\_\{i\}=1\+\\deltaand simplifying∂ℒ/∂σi\\partial\\mathcal\{L\}/\\partial\\sigma\_\{i\}yields1/σi−σi≈−2​δ\+O​\(δ2\)1/\\sigma\_\{i\}\-\\sigma\_\{i\}\\approx\-2\\delta\+O\(\\delta^\{2\}\), so∂ℒ/∂σi≈−2​δ\\partial\\mathcal\{L\}/\\partial\\sigma\_\{i\}\\approx\-2\\delta\. Thus, the collapse fixed pointσi∗=1\\sigma\_\{i\}^\{\*\}=1is a*locally asymptotically stable point with linearisation rate−2\-2*\. Forgiσ\>0g\_\{i\}^\{\\sigma\}\>0, the gradient ratio is the necessary and sufficient collapse indicator stated in Proposition[1](https://arxiv.org/html/2607.05531#Thmprop1)\.

ρi=\|giσ\|\|1/σi−σi\|\.\\rho\_\{i\}\\;=\\;\\frac\{\|g\_\{i\}^\{\\sigma\}\|\}\{\|1/\\sigma\_\{i\}\-\\sigma\_\{i\}\|\}\.\(A\.3\)

#### Proof of sufficiency:

Assumeρi​\(t\)<1\\rho\_\{i\}\(t\)<1for allt≥T0t\\geq T\_\{0\}\. Then\|giσ\|<\|1/σi−σi\|\|g\_\{i\}^\{\\sigma\}\|<\|1/\\sigma\_\{i\}\-\\sigma\_\{i\}\|, so∂ℒ/∂σi\>0\\partial\\mathcal\{L\}/\\partial\\sigma\_\{i\}\>0wheneverσi<1\\sigma\_\{i\}<1\. The sequence\{σi​\(t\)\}t≥T0\\\{\\sigma\_\{i\}\(t\)\\\}\_\{t\\geq T\_\{0\}\}is monotonically non\-decreasing and bounded above by11\. By the*Monotone Convergence Theorem*it converges to a limitσi∞≤1\\sigma\_\{i\}^\{\\infty\}\\leq 1\. Ifσi∞<1\\sigma\_\{i\}^\{\\infty\}<1, then1/σi∞−σi∞\>01/\\sigma\_\{i\}^\{\\infty\}\-\\sigma\_\{i\}^\{\\infty\}\>0, and Eq\.[A\.1](https://arxiv.org/html/2607.05531#A1.Ex1)requiresgiσ​\(σi∞\)=0g\_\{i\}^\{\\sigma\}\(\\sigma\_\{i\}^\{\\infty\}\)=0at a fixed point, which forcesσi∞=1\\sigma\_\{i\}^\{\\infty\}=1, which is a contradiction\. Henceσi∞=1\\sigma\_\{i\}^\{\\infty\}=1\.■\\blacksquare

#### Proof of necessity:

Under the decay condition provided in the proposition, nearσi=1\\sigma\_\{i\}=1the𝕂​𝕃\\mathbb\{KL\}part is written as

\|1σi−σi\|=\(1−σi\)​\(1\+σi\)σi∼2​\(1−σi\)as​σi→1−,\\left\|\\frac\{1\}\{\\sigma\_\{i\}\}\-\\sigma\_\{i\}\\right\|\\;=\\;\\frac\{\(1\-\\sigma\_\{i\}\)\(1\+\\sigma\_\{i\}\)\}\{\\sigma\_\{i\}\}\\;\\sim\\;2\(1\-\\sigma\_\{i\}\)\\quad\\text\{as \}\\sigma\_\{i\}\\to 1^\{\-\},\(A\.4\)while the decay bound\|giσ\|=O​\(\(1−σi\)α\)\|g\_\{i\}^\{\\sigma\}\|=O\\\!\\left\(\(1\-\\sigma\_\{i\}\)^\{\\alpha\}\\right\)gives the ratio bounded by

ρi=O​\(\(1−σi\)α\)2​\(1−σi\)=O​\(\(1−σi\)α−1\)\.\\rho\_\{i\}\\;=\\;\\frac\{O\\\!\\left\(\(1\-\\sigma\_\{i\}\)^\{\\alpha\}\\right\)\}\{2\(1\-\\sigma\_\{i\}\)\}\\;=\\;O\\\!\\left\(\(1\-\\sigma\_\{i\}\)^\{\\alpha\-1\}\\right\)\.\(A\.5\)- •α\>1\\alpha\>1:\(1−σi\)α−1→0\(1\-\\sigma\_\{i\}\)^\{\\alpha\-1\}\\to 0asσi→1−\\sigma\_\{i\}\\to 1^\{\-\}, soρi→0\\rho\_\{i\}\\to 0\. Collapse impliesρi→0\\rho\_\{i\}\\to 0\. Hence,*The ratio is anecessary and sufficientcollapse signal for dimensionii*\.■\\blacksquare
- •α=1\\alpha=1:ρi→g0/2\>0\\rho\_\{i\}\\to g\_\{0\}/2\>0asσi→1\\sigma\_\{i\}\\to 1: the ratio stays bounded away from zero regardless of whether collapse occurs, giving a potential*false negative*\. The conditionα\>1\\alpha\>1in the proposition is therefore not a restriction but a*regime identifier*; it characterises the decoders for whichρi\\rho\_\{i\}is an informative diagnostic\.
- •α<1\\alpha<1:ρi→∞\\rho\_\{i\}\\to\\infty; the decoder resists collapse\. No collapse occurs\.

### A\.2 Information Gap \(Proposition[2](https://arxiv.org/html/2607.05531#Thmprop2)\)

We derive the gapΔ​I=I​\(X;ϕ\)−I​\(X;Z\)≥0\\Delta I=I\(X;\\phi\)\-I\(X;Z\)\\geq 0under Gaussian posteriors and show it attenuates change in the encoder\. By the*Data Processing Inequality*Cover \([1999](https://arxiv.org/html/2607.05531#bib.bib62)\), we have

I​\(X;Z\)≤I​\(X;ϕ\)⟹Δ​I=I​\(X;ϕ\)−I​\(X;Z\)≥0\.I\(X;Z\)\\;\\leq\\;I\(X;\\phi\)\\qquad\\Longrightarrow\\qquad\\Delta I\\;=\\;I\(X;\\phi\)\-I\(X;Z\)\\;\\geq\\;0\.\(20\)Sincezi=μi\+σi​ϵiz\_\{i\}=\\mu\_\{i\}\+\\sigma\_\{i\}\\epsilon\_\{i\}withσi\>0\\sigma\_\{i\}\>0is not invertible \(givenzi=zz\_\{i\}=z, we cannot recoverϵi\\epsilon\_\{i\}withoutμi,σi\\mu\_\{i\},\\sigma\_\{i\}\), the map is strictly lossy:Δ​I\>0\\Delta I\>0for allσi\>0\\sigma\_\{i\}\>0\. Equivalently, by the chain rule for mutual information, the gap satisfies

Δ​I=I​\(X;ϕ∣Z\)≥0,\\Delta I\\;=\\;I\\\!\\left\(X;\\phi\\mid Z\\right\)\\;\\geq\\;0,\(A\.6\)the conditional mutual information betweenXXand the encoder parameters givenZZ\.

#### Information gap under Gaussian posteriors:

Consider an approximate posterior estimateq​\(zi\|x\)=𝒩​\(μi​\(x\),σi2​\(x\)\)q\(z\_\{i\}\|x\)=\\mathcal\{N\}\(\\mu\_\{i\}\(x\),\\sigma\_\{i\}^\{2\}\(x\)\), and its conditional entropy

H​\[Zi\|X\]=12​log⁡\(2​π​e\)\+12​𝔼p​\(x\)​\[log⁡σi2​\(x\)\]\.H\[Z\_\{i\}\|X\]\\;=\\;\\tfrac\{1\}\{2\}\\log\(2\\pi e\)\+\\tfrac\{1\}\{2\}\\mathbb\{E\}\_\{p\(x\)\}\\\!\\left\[\\log\\sigma\_\{i\}^\{2\}\(x\)\\right\]\.\(A\.7\)SinceI​\(X;ϕ\)=H​\[ϕ\]I\(X;\\phi\)=H\[\\phi\]is fixed by the encoder weights \(encoder is deterministic\), changes in the gapΔ​I\\Delta Iare driven by changes inI​\(X;Z\)I\(X;Z\)\. Under approximate marginal matchingq​\(z\)≈𝒩​\(𝟎,𝐈\)q\(z\)\\approx\\mathcal\{N\}\(\\mathbf\{0\},\\mathbf\{I\}\)\(valid under optimized𝕂​𝕃\\mathbb\{KL\}regularization\), the accessible mutual information reduces to

I​\(X;Z\)≈12​∑i=1d𝔼p​\(x\)​\[−log⁡σi2​\(x\)\]≥0\.I\(X;Z\)\\;\\approx\\;\\frac\{1\}\{2\}\\sum\_\{i=1\}^\{d\}\\mathbb\{E\}\_\{p\(x\)\}\\\!\\left\[\-\\log\\sigma\_\{i\}^\{2\}\(x\)\\right\]\\;\\geq\\;0\.\(A\.8\)The gap grows monotonically:∂\(Δ​I\)/∂σi=1/σi\>0\\partial\(\\Delta I\)/\\partial\\sigma\_\{i\}=1/\\sigma\_\{i\}\>0\. At collapse \(σi→1\\sigma\_\{i\}\\to 1\), dimensionii’s gap reaches its maximumI​\(X;ϕi\)I\(X;\\phi\_\{i\}\)whileI​\(X;Zi\)≈−12​log⁡σi2→0I\(X;Z\_\{i\}\)\\approx\-\\tfrac\{1\}\{2\}\\log\\sigma\_\{i\}^\{2\}\\to 0, reflecting complete information loss\.

#### Derivation of the sensitivity equation:

From the definition, we have the identity

I​\(X;Z\)=I​\(X;ϕ\)−Δ​I\.I\(X;Z\)\\;=\\;I\(X;\\phi\)\-\\Delta I\.\(A\.9\)TreatingΔ​I\\Delta Ias a function ofI​\(X;ϕ\)I\(X;\\phi\)and differentiating Eq\.[A\.9](https://arxiv.org/html/2607.05531#A1.Ex9)along the training trajectory

∂I​\(X;Z\)∂I​\(X;ϕ\)=1−∂Δ​I∂I​\(X;ϕ\)\.\\frac\{\\partial\\,I\(X;Z\)\}\{\\partial\\,I\(X;\\phi\)\}\\;=\\;1\-\\frac\{\\partial\\,\\Delta I\}\{\\partial\\,I\(X;\\phi\)\}\.\(A\.10\)Sinceℛ=f​\(I​\(X;Z\)\)\\mathcal\{R\}=f\(I\(X;Z\)\)for monotonically increasingff, the chain rule gives

∂ℛ∂I​\(X;ϕ\)=∂ℛ∂I​\(X;Z\)⋅\(1−∂Δ​I∂I​\(X;ϕ\)\)\.\\frac\{\\partial\\mathcal\{R\}\}\{\\partial\\,I\(X;\\phi\)\}\\;=\\;\\frac\{\\partial\\mathcal\{R\}\}\{\\partial\\,I\(X;Z\)\}\\cdot\\left\(1\-\\frac\{\\partial\\,\\Delta I\}\{\\partial\\,I\(X;\\phi\)\}\\right\)\.\(A\.11\)This is equation Eq\.[6](https://arxiv.org/html/2607.05531#S3.E6)in the main text\.

#### Duality with marginal mismatch:

From the standard ELBO decomposition\(Hoffman and Johnson,[2016](https://arxiv.org/html/2607.05531#bib.bib27)\),

𝕂​𝕃​\(q​\(z\|x\)∥p​\(z\)\)=I​\(X;Z\)\+𝕂​𝕃​\(q​\(z\)∥p​\(z\)\)\.\\displaystyle\\mathbb\{KL\}\(q\(z\|x\)\\\|p\(z\)\)\\;=\\;I\(X;Z\)\+\\mathbb\{KL\}\(q\(z\)\\\|p\(z\)\)\.\(A\.12\)Substituting Eq\.[A\.9](https://arxiv.org/html/2607.05531#A1.Ex9):

𝕂​𝕃​\(q​\(z\)∥p​\(z\)\)=𝕂​𝕃​\(q​\(z\|x\)∥p​\(z\)\)−I​\(X;ϕ\)\+Δ​I\.\\displaystyle\\mathbb\{KL\}\(q\(z\)\\\|p\(z\)\)\\;=\\;\\mathbb\{KL\}\(q\(z\|x\)\\\|p\(z\)\)\-I\(X;\\phi\)\+\\Delta I\.\(A\.13\)

## Appendix B

### B\.1 Equilibrium Variance Targetσ∗=λ−1/\(λ−1\)\\sigma^\{\*\}=\\lambda^\{\-1/\(\\lambda\-1\)\}

Underλ\\lambda\-scaling, the reparameterization iszi=μi\+σiλ​ϵiz\_\{i\}=\\mu\_\{i\}\+\\sigma\_\{i\}^\{\\lambda\}\\epsilon\_\{i\}\. Differentiating the ELBO objective with respect tolog⁡σi\\log\\sigma\_\{i\}, the modification introduced byλ\\lambda\-scaling relative to standard sampling produces the following scaling term

Δ​∇i:=∂ℒλ∂log⁡σi−∂ℒ1∂log⁡σi=\(λ​σiλ−σi\)⋅giσ,\\displaystyle\\Delta\\nabla\_\{i\}:=\\frac\{\\partial\\mathcal\{L\}\_\{\\lambda\}\}\{\\partial\\log\\sigma\_\{i\}\}\-\\frac\{\\partial\\mathcal\{L\}\_\{1\}\}\{\\partial\\log\\sigma\_\{i\}\}\\;=\\;\\bigl\(\\lambda\\,\\sigma\_\{i\}^\{\\lambda\}\-\\sigma\_\{i\}\\bigr\)\\cdot g\_\{i\}^\{\\sigma\},\(21\)wheregiσ:=𝔼ϵi​\[∇zilog⁡pθ​\(x\|z\)⋅ϵi\]g\_\{i\}^\{\\sigma\}:=\\mathbb\{E\}\_\{\\epsilon\_\{i\}\}\[\\nabla\_\{z\_\{i\}\}\\log p\_\{\\theta\}\(x\|z\)\\cdot\\epsilon\_\{i\}\]is the reconstruction gradient\. The derivation follows from the chain rule throughσiλ\\sigma\_\{i\}^\{\\lambda\}:∂\(σiλ​ϵi\)/∂log⁡σi=λ​σiλ\\partial\(\\sigma\_\{i\}^\{\\lambda\}\\epsilon\_\{i\}\)/\\partial\\log\\sigma\_\{i\}=\\lambda\\sigma\_\{i\}^\{\\lambda\}, compared to∂\(σi​ϵi\)/∂log⁡σi=σi\\partial\(\\sigma\_\{i\}\\epsilon\_\{i\}\)/\\partial\\log\\sigma\_\{i\}=\\sigma\_\{i\}in the standard case\.

The scaling term in Eq\.[21](https://arxiv.org/html/2607.05531#Ax1.E21)is zero iffλ​σiλ=σi\\lambda\\sigma\_\{i\}^\{\\lambda\}=\\sigma\_\{i\}, which gives the equilibrium condition forσi\\sigma\_\{i\}:

λ​σiλ−1\\displaystyle\\lambda\\,\\sigma\_\{i\}^\{\\lambda\-1\}=1\\displaystyle\\;=\\;1σi∗\\displaystyle\\sigma\_\{i\}^\{\*\}=λ−1/\(λ−1\)\.\\displaystyle\\;=\\;\\lambda^\{\-1/\(\\lambda\-1\)\}\.\(22\)The above is valid forλ\>1\\lambda\>1\.

#### σ∗\\sigma^\{\*\}is a stable attractor:

The scaling term Eq\.[21](https://arxiv.org/html/2607.05531#Ax1.E21)acts as a restoring force\. Forgiσ\>0g\_\{i\}^\{\\sigma\}\>0, the following conditions hold

- •σi<σ∗\\sigma\_\{i\}<\\sigma^\{\*\}: thenσiλ−1<1/λ\\sigma\_\{i\}^\{\\lambda\-1\}<1/\\lambda, soλ​σiλ<σi\\lambda\\sigma\_\{i\}^\{\\lambda\}<\\sigma\_\{i\}, givingΔ​∇i<0\\Delta\\nabla\_\{i\}<0\. The gradient onlog⁡σi\\log\\sigma\_\{i\}is negative:σi\\sigma\_\{i\}is pushed*upward*towardσ∗\\sigma^\{\*\}\.
- •σi\>σ∗\\sigma\_\{i\}\>\\sigma^\{\*\}: thenσiλ−1\>1/λ\\sigma\_\{i\}^\{\\lambda\-1\}\>1/\\lambda, soλ​σiλ\>σi\\lambda\\sigma\_\{i\}^\{\\lambda\}\>\\sigma\_\{i\}, givingΔ​∇i\>0\\Delta\\nabla\_\{i\}\>0\. The gradient onlog⁡σi\\log\\sigma\_\{i\}is positive:σi\\sigma\_\{i\}is pushed*downward*towardσ∗\\sigma^\{\*\}\.

The force is bidirectional and vanishes only atσ∗\\sigma^\{\*\}:σ∗\\sigma^\{\*\}is a stable fixed point\. Every dimension, regardless of its initial value, is driven toward the*same*target\. See Figure[2\(a\)](https://arxiv.org/html/2607.05531#S4.F2.sf1)and Figure[4](https://arxiv.org/html/2607.05531#S6.F4)for illustrative and empirical data of this behaviour\.

### B\.2 Optimal Per\-Dimensionλi∗\\lambda^\{\*\}\_\{i\}

#### Objective:

For a single latent dimensioniiwith encoder varianceσi∈\(0,1\)\\sigma\_\{i\}\\in\(0,1\), the per\-dimension net information gain objective \(Eq\.[15](https://arxiv.org/html/2607.05531#S4.E15)in the main text\) is:

𝒥​\(λi,σi\)=\(λi−1\)​\|log⁡σi\|−δ​𝕂​𝕃​\(λi\),δ\>1,\\mathcal\{J\}\(\\lambda\_\{i\},\\sigma\_\{i\}\)\\;=\\;\(\\lambda\_\{i\}\-1\)\\,\|\\\!\\log\\sigma\_\{i\}\|\\;\-\\;\\delta\\,\\mathbb\{KL\}\(\\lambda\_\{i\}\),\\qquad\\delta\>1,\(23\)where\|log⁡σi\|=−log⁡σi\>0\|\\log\\sigma\_\{i\}\|=\-\\log\\sigma\_\{i\}\>0forσi∈\(0,1\)\\sigma\_\{i\}\\in\(0,1\), and the marginal𝕂​𝕃\\mathbb\{KL\}cost \(Eq\.[14](https://arxiv.org/html/2607.05531#S4.E14)in the main text\) is:

𝕂​𝕃​\(λi\)=12​\(σi2​λi−σi2−log⁡σi2​λi\+log⁡σi2\)\.\\mathbb\{KL\}\(\\lambda\_\{i\}\)\\;=\\;\\frac\{1\}\{2\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-\\sigma\_\{i\}^\{2\}\-\\log\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\+\\log\\sigma\_\{i\}^\{2\}\\right\)\.\(24\)

#### Derivative of𝒥\\mathcal\{J\}with respect toλi\\lambda\_\{i\}:

The information gain term contributes\|log⁡σi\|\|\\log\\sigma\_\{i\}\|, which is constant inλi\\lambda\_\{i\}\. For the marginal cost, the onlyλi\\lambda\_\{i\}\-dependent terms in Eq\.[24](https://arxiv.org/html/2607.05531#Ax1.E24)areσi2​λi\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}andlog⁡σi2​λi=2​λi​log⁡σi\\log\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}=2\\lambda\_\{i\}\\log\\sigma\_\{i\}\. Hence,

∂𝕂​𝕃∂λi\\displaystyle\\frac\{\\partial\\mathbb\{KL\}\}\{\\partial\\lambda\_\{i\}\}=12​\(2​σi2​λi​log⁡σi−2​log⁡σi\)=log⁡σi​\(σi2​λi−1\)\.\\displaystyle\\;=\\;\\frac\{1\}\{2\}\\\!\\left\(2\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\\log\\sigma\_\{i\}\-2\\log\\sigma\_\{i\}\\right\)\\;=\\;\\log\\sigma\_\{i\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\\right\)\.\(25\)Forσi∈\(0,1\)\\sigma\_\{i\}\\in\(0,1\):log⁡σi<0\\log\\sigma\_\{i\}<0andσi2​λi<1\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}<1, soσi2​λi−1<0\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1<0\. Therefore∂𝕂​𝕃/∂λi=log⁡σi​\(σi2​λi−1\)\>0\\partial\\mathbb\{KL\}/\\partial\\lambda\_\{i\}=\\log\\sigma\_\{i\}\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\)\>0: the marginal cost is strictly increasing inλi\\lambda\_\{i\}\. The full derivative of the objective is

∂𝒥∂λi=\|log⁡σi\|−δ​log⁡σi​\(σi2​λi−1\)\.\\frac\{\\partial\\mathcal\{J\}\}\{\\partial\\lambda\_\{i\}\}\\;=\\;\|\\log\\sigma\_\{i\}\|\-\\delta\\log\\sigma\_\{i\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\\right\)\.\(26\)

#### Stationary\-point condition:

Setting Eq\.[26](https://arxiv.org/html/2607.05531#Ax1.E26)to zero,

\|log⁡σi\|=δ​log⁡σi​\(σi2​λi−1\)\.\|\\log\\sigma\_\{i\}\|\\;=\\;\\delta\\log\\sigma\_\{i\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\\right\)\.\(27\)Substituting\|log⁡σi\|=−log⁡σi\|\\log\\sigma\_\{i\}\|=\-\\log\\sigma\_\{i\}\(sinceσi<1\\sigma\_\{i\}<1\):

−log⁡σi=δ​log⁡σi​\(σi2​λi−1\)\.\-\\log\\sigma\_\{i\}\\;=\\;\\delta\\log\\sigma\_\{i\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\\right\)\.\(28\)Dividing both sides bylog⁡σi≠0\\log\\sigma\_\{i\}\\neq 0:

−1=δ​\(σi2​λi−1\)\.\-1\\;=\\;\\delta\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\\right\)\.\(29\)Rearranging,

σi2​λi=1−1δ\.\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\\;=\\;1\-\\frac\{1\}\{\\delta\}\.\(30\)

#### Solving forλi∗\\lambda\_\{i\}^\{\*\}:

Taking the natural logarithm of both sides of Eq\.[30](https://arxiv.org/html/2607.05531#Ax1.E30):

2​λi∗​log⁡σi=log⁡\(1−1δ\)\.2\\lambda\_\{i\}^\{\*\}\\,\\log\\sigma\_\{i\}\\;=\\;\\log\\\!\\left\(1\-\\frac\{1\}\{\\delta\}\\right\)\.\(31\)Solving forλi∗\\lambda\_\{i\}^\{\*\}:

λi∗=log⁡\(1−1/δ\)2​log⁡σi\.\\lambda\_\{i\}^\{\*\}\\;=\\;\\frac\{\\log\(1\-1/\\delta\)\}\{2\\log\\sigma\_\{i\}\}\.\(32\)Forσi∈\(0,1\)\\sigma\_\{i\}\\in\(0,1\):log⁡σi<0\\log\\sigma\_\{i\}<0\. Forδ\>1\\delta\>1:1−1/δ∈\(0,1\)1\-1/\\delta\\in\(0,1\), solog⁡\(1−1/δ\)<0\\log\(1\-1/\\delta\)<0\. Thereforeλi∗\>0\\lambda\_\{i\}^\{\*\}\>0\. Enforcing the constraintλi≥1\\lambda\_\{i\}\\geq 1:

λi∗=max\(1,log⁡\(1−1/δ\)2​log⁡σi\)\.\\boxed\{\\lambda\_\{i\}^\{\*\}\\;=\\;\\max\\\!\\left\(1,\\;\\frac\{\\log\(1\-1/\\delta\)\}\{2\\log\\sigma\_\{i\}\}\\right\)\.\}\(33\)

#### The stationary point is a global maximum:

The second derivative of Eq\.[23](https://arxiv.org/html/2607.05531#Ax1.E23):

∂2𝒥∂λi2=−δ​∂2𝕂​𝕃∂λi2\.\\frac\{\\partial^\{2\}\\mathcal\{J\}\}\{\\partial\\lambda\_\{i\}^\{2\}\}\\;=\\;\-\\delta\\,\\frac\{\\partial^\{2\}\\mathbb\{KL\}\}\{\\partial\\lambda\_\{i\}^\{2\}\}\.\(34\)From Eq\.[25](https://arxiv.org/html/2607.05531#Ax1.E25):

∂2𝕂​𝕃∂λi2=∂∂λi​\[log⁡σi​\(σi2​λi−1\)\]=2​\(log⁡σi\)2​σi2​λi\>0\.\\frac\{\\partial^\{2\}\\mathbb\{KL\}\}\{\\partial\\lambda\_\{i\}^\{2\}\}\\;=\\;\\frac\{\\partial\}\{\\partial\\lambda\_\{i\}\}\\\!\\left\[\\log\\sigma\_\{i\}\\\!\\left\(\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\-1\\right\)\\right\]\\;=\\;2\\,\(\\log\\sigma\_\{i\}\)^\{2\}\\,\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\\;\>\\;0\.\(35\)Therefore:

∂2𝒥∂λi2=−2​δ​\(log⁡σi\)2​σi2​λi<0\.\\frac\{\\partial^\{2\}\\mathcal\{J\}\}\{\\partial\\lambda\_\{i\}^\{2\}\}\\;=\\;\-2\\delta\\,\(\\log\\sigma\_\{i\}\)^\{2\}\\,\\sigma\_\{i\}^\{2\\lambda\_\{i\}\}\\;<\\;0\.\(36\)The objective is strictly concave inλi\\lambda\_\{i\}, confirming that the stationary point Eq\.[33](https://arxiv.org/html/2607.05531#Ax1.E33)is a global maximum\. Hence, maximizing the net information gain for a givenδ\\deltahyperparameter\.