gate-normalization

Tag

#gate-normalization

@Jianlin_S: MoE (9): The Gate Normalization Debate https://kexue.fm/archives/11782

X AI KOLs Timeline ↗ · 2d ago

A blog post discussing the debate on gate normalization in Mixture of Experts (MoE) models.

0 favorites 0 likes