Tag
The paper introduces correlated expert placement and token shuffling to exploit router-learned expert correlations, cutting all-to-all communication time by 1.16–2.63× and end-to-end MoE pretraining step time by up to 1.41× in Megatron-LM without altering routing decisions or model parameters.
Presents SNI-GNN, a SmartNIC-assisted full-graph GNN training system that reduces inter-node communication by predicting remote embeddings in-network, achieving 1.3–3.6x speedups with negligible accuracy loss.