training-systems

HyScale 提出了一种面向长上下文训练的学习型稀疏注意力系统，通过 HyTopK（免分数表的精确 top-k 选择）和选择重放（selection replay）技术，在 1M token 序列上将 GLM-5.1 训练速度提升至 Megatron-LM 的 2.07 倍。

The author shares experiences and insights from nnScaler to large-scale distributed training systems, discussing correctness, flexibility, boundary expansion, and the challenges brought by post-training and reinforcement learning.

LLMSys-PaperList is a curated reading list on GitHub that organizes LLM systems research papers and resources into practical categories such as training systems, serving systems, and multi-modal coverage, helping AI/ML engineers and researchers stay updated.