sign-methods

Tag

Cards List
#sign-methods

SignMuon: Communication-Efficient Distributed Muon Optimization

arXiv cs.LG · 2026-05-19 Cached

SignMuon is a 1-bit, matrix-aware optimizer for distributed training that combines signSGD's majority-vote sign aggregation with Muon's polar-step framework, achieving 32x bandwidth reduction over float32 while maintaining strong convergence and performance on benchmarks like CIFAR-10/ResNet-50 and nanoGPT.

0 favorites 0 likes
← Back to home

Submit Feedback