skewadam

Tag

Cards List
#skewadam

SkewAdam: A tiered optimizer that cuts MoE state memory by 97% (fits a 6.7B MoE on a 40GB GPU) [R]

Reddit r/MachineLearning ↗ · 2026-07-22

SkewAdam is a tiered optimizer that reduces memory usage for MoE state by 97%, enabling a 6.7B MoE model to fit on a single 40GB GPU.

0 favorites 0 likes
← Back to home

Submit Feedback