Tag
The author shares experiences and insights from nnScaler to large-scale distributed training systems, discussing correctness, flexibility, boundary expansion, and the challenges brought by post-training and reinforcement learning.