Tag
This paper proposes a method to decompose the response distribution of language models into structured, strategy-conditioned representations using a latent variable, addressing posterior collapse with a model-directed reconstruction objective.
Identifies two coupled causes of posterior collapse in VAEs and introduces λ-VAE, a modification to the reparameterization step that equalizes variance across latent dimensions, reducing collapse and improving information capacity.