Tag
Reveals that MoE routing in LLMs follows a Frequency-Diversity Law analogous to Huffman coding, and proposes Subset Difference Pruning to eliminate functional redundancy and achieve optimal compression.