Tag
vLLM introduces a native sharded weight transfer engine using Ray Direct Transport (RDT) for efficient weight syncing in large-scale online RL setups, enhancing performance for models like Kimi K2.
This paper tackles code generation for no-resource programming languages by building benchmarks and proposing a method that combines further pre-training with weight difference transfer to create specialized instruction-following models at reduced cost.
This paper proposes a method to convert pretrained Softmax attention models into linear-complexity Test-Time Training (TTT) architectures, achieving comparable text-to-image quality to fine-tuned Softmax models while significantly accelerating inference. The approach is validated by linearizing Stable Diffusion 3.5, resulting in SD3.5-T^5 with 1.32x speedup at 1K resolution.