systems-co-design

SpecFold 论文提出一种算法-系统协同设计，通过识别并折叠多分支投机验证中的计算冗余（token级残差门控与折叠注意力/FFN复用），在扩散大语言模型上实现最高1.64x吞吐提升（相比Spiffy），且保持任务性能相当。

This paper introduces MOSAIC, a framework that jointly optimizes sparse Mixture-of-Experts model architecture and hardware systems for large-scale pretraining, showing that compute-optimal sparsity is not necessarily cluster-optimal when MFU, communication costs, and parallel layouts are considered.