Tag
FLAMIA 提出了一族面向 NVIDIA Blackwell GPU 的混合精度注意力内核，利用 MXFP8 与 NVFP4/FP8 路径，在避免因果信息泄漏的同时加速 Transformer 训练，在 B200/B300 上实现约 1.06–1.42 倍的前向与反向注意力加速。
Metis-1.6 technical report introduces Praxis (2.94B) and Logos (10.23B), two MoE language models with a hybrid Mamba-2/attention stack built by a single researcher on ~1.02T tokens, achieving Pareto-efficient math performance while openly detailing training failures and fixes.
TrainBridge is an agentic framework that transfers GPU compilation optimization guidance to Ascend NPU training programs, achieving an average 1.7× speedup across four workloads and nearly 2.0× across 27 Modded-NanoGPT speedrun records by extracting optimization intents from GPU executions and iteratively refining NPU programs through profiling.
This paper measures exactly how much weight gradient is lost when training looped transformers via truncated backpropagation across nine models, finding that the lost gradient is largely set near the output, diminishes during fine-tuning, and does not clearly outperform measurement-free cost heuristics like Parcae's rule.
该论文研究了在均匀扩散语言模型（UDLM）中循环早期 Transformer 层的方法，在不改变训练目标和采样器的前提下，以显著更少的训练 FLOPs 达到相同的验证 NLL，并在推理时降低生成困惑度。
The paper introduces Future Chunk Prediction (FCP), an auxiliary objective for autoregressive video diffusion models that predicts temporally separated chunks without intervening visual context, improving physical plausibility and reducing training compute. The approach requires no additional pretrained visual encoder and hints at more structured representation spaces useful for autoregressive world models.
A scaling study over ~600 training runs finds that full BPTT on looped transformers outperforms truncated alternatives at matched compute, with two fully trained loops emerging as the compute-optimal setting that beats non-looped models with ~27% fewer parameters.
SWIFT 提出一种自引导的弱到强扩散模型训练方法，通过将回归目标外推偏离冻结的弱去噪器来降低误差和训练成本，且不增加推理开销。实验在 CIFAR-10 和 ImageNet-256 上验证，ImageNet-256 某配置以约一半训练算力超越基线最佳 FID，RAEv2 上无引导达到 FID 1.15。
FlashMask-3 introduces a Hopper-optimized mask-aware sparse attention kernel and zero-communication distributed mask localization for long-context training, achieving major TFLOPs and throughput gains over FlashMask, FlexAttention, MagiAttention, and Megatron-Core baselines, including 7.3-11.0% end-to-end speedup on a 118B MoE trained on 128 H100 GPUs at 128K context.
Minute-scale training for microrobot navigation，一篇发表于 Nature Machine Intelligence 的研究，探讨如何在数分钟内完成微型机器人导航策略的训练。
该论文提出 Elastic Expert Routing，通过从以 k 为中心的局部离散分布中随机采样激活专家数量，平滑传统 top-k 路由的硬边界，在保持计算成本不变的前提下提升稀疏 MoE 模型在微调与从头预训练场景下的下游表现。
EDiS 提出一个边缘不相交子图稀疏化框架，将一次性结构提取与逐轮图组合分离，从而在降低 GNN 稀疏训练开销的同时保持拓扑多样性，并在 19 个基准上取得最佳平均成绩。
论文提出 Self-Retrospection Distillation (SRD)，通过让策略在行动前预测可能遇到的陷阱，并用事后轨迹的自蒸馏对齐前瞻预测，解决了 GRPO 类方法中大量全同奖励组导致零梯度的问题；在 2B 模型 98% 全失败组的场景下，GRPO+SRD 将训练成功率从 0% 提升至 60.6%。
The paper proposes ACES (Adaptive Coverage-aware Efficient Sampling), a structured sampling framework for implicit neural representations that decouples domain coverage from importance weighting via adaptive spatial partitions, achieving faster convergence and lower error on scientific field learning tasks.
Memorizon decouples long-span supervision from attention in streaming world models by having each scored chunk retrieve a shared bounded top-K latent bank via camera co-visibility, enabling consistent revisits beyond the context window at minimal extra cost. Retrieval improves revisit consistency over a sliding-window baseline by up to 30%, and results show the model genuinely uses the retrieved latents.
The paper introduces CARE, a condition-aware representation regularization framework for diffusion models that improves sample quality and training efficiency by dynamically modulating feature distributions based on condition similarity. Empirically, it achieves significant reductions in FID and faster convergence for both class-to-image and text-to-image tasks.
Linum AI introduces JiT-DDT, a novel encoder-decoder architecture that trains text-to-image models 3.6× faster than previous methods while generating images with higher resolution.
Xing4.0-29B-A4B is an open-source 29B-parameter large language model optimized for agent tasks and Ascend NPU, featuring a MoE architecture and achieving high training efficiency with competitive benchmark results.
FlashREINFORCE introduces a critic-free, single-rollout asynchronous reinforcement learning framework for agentic language models, improving stability and efficiency by learning from each trajectory immediately.
LeanGRPO eliminates redundant recomputation in diffusion RL methods by introducing recompute-free training schedules, achieving up to 1.83x speedup while preserving optimization objectives.