Tag
The paper introduces Hierarchical Copula-Gumbel-Top-K (H-CGA) routing, a method to control joint dependence among token routing choices in frozen Mixture-of-Experts models while keeping each token's routing law exactly fixed. It provides theoretical trade-offs between coherence and load dispersion and validates the mechanism with a small-scale pilot.
A discussion on Sam Altman's vision of AI as a metered utility like electricity or water, raising questions about practicality and concerns over dependence on a few companies.