Tag
This paper proposes a unified approach to interpret knowledge distillation in LLMs using game-theoretic interactions, discovering that distillation sparsifies interactions, and introduces a loss function CIP to improve performance.