Tag
This paper introduces Canopy Entropy (CE⋆) to measure the effective size of the generation space in language models, and finds that fine-tuning reorganizes uncertainty into more informative and semantically meaningful outputs, nearly tripling the correlation between entropy rate and semantic diversity.