Tag
FARO is a function-space adaptive optimizer that assigns each parameter block a geometry based on its counterfactual return and reallocates the per-block update budget via cross-block equal risk contribution, matching hand-designed recipes like NorMuonAndAdam while providing robust small loss gains on GPT-style training.
Carnegie Mellon paper shows that pretraining, fine-tuning, and RL can collapse into a single training rule (MaxRL as likelihood training on correct answers), with blended data runs outperforming staged training on reasoning and coding benchmarks while preserving general skills.
Modal announced first-class support for post-training Reflection's Beam model and released Modal Dojo, an open-source library and skill bundle for production-grade LLM training (including RL) on Modal, with an example training Qwen3.5-4B on GSM8K.
A thread presenting research that fits scaling laws to determine how much compute to invest in synthetic rewrites for improving data quality, and whether a model trained on rewrites can surpass the rewriting model itself.
River API now supports training LLMs with arbitrary custom loss functions written in PyTorch, which are traced locally, sent as a graph, and executed at scale on River's remote GPU cluster with parallelization and memory-efficient streaming.
Wikimedia Foundation filed an amicus brief arguing that courts should protect LLM training on copyrighted work while still curbing chatbots from reproducing that content, warning that overly broad rulings in cases involving OpenAI, Microsoft, and The New York Times could threaten the open web.
The article unpacks the "Reinforcement Learning with Self-Feedback" technique mentioned in the DeepSeek V4.1 flash update, explains how it provides denser reward signals for long-horizon agent tasks that lack ground truth answers — such as "wish-based programming" — and compiles a list of 10 papers on self-feedback reinforcement learning from 2024–2026.
At EMNLP 2026, T-Tech researchers presented LIMe, a lightweight attention extension that reuses key-value buffers from earlier layers to mitigate representation collapse, reducing LLM training compute by ~15% at equal quality versus baseline LLaMA on ~1B-parameter models.
This paper analyzes the training-inference mismatch in on-policy RL for LLMs, decomposing sampled-token error into numerical discrepancy and policy lag, and derives a discrepancy-lag correction (DLC) that centers errors, clips lag ratios, and resynchronizes the sampler to prevent training collapses.
论文提出 InfraDev 基准，包含 36 个源自 Megatron-LM、vLLM 和 verl 真实 PR 的任务，用于评估编码代理开发生产级 AI 基础设施的能力。目前最佳前沿模型仅解决 38.9% 的任务，揭示了代理在分布式系统开发中的显著短板。
Yale researchers establish causal upper and lower bounds on CoT monitorability metrics, showing that the minimal-criterion TPR metric can overestimate intervention-caused behavior detection, and propose three intervention-aware post-training methods (off-policy self-distillation, RL, on-policy self-distillation) that improve LLM reasoning-trace monitorability.
This paper analyzes when and how the Muon optimizer provides preconditioning benefits, showing it offers no acceleration under isotropic noise but can reduce task-frequency dependence from linear to square-root under structured mini-batch gradient noise, while avoiding SignSGD's rotation-induced negative transfer.
LazyFlash 提出一种前向稠密、后向稀疏的注意力机制，通过梯度感知的块稀疏化在保持模型性能的同时，实现最高 5.7x 的注意力反向加速和 1.7x 的端到端训练加速，适用于长上下文 LLM 微调。
Modal 发布 Spindle，一个开源、与 Tinker 兼容的训练服务器，可在 Modal 工作区中运行，让现有 Tinker 脚本无需改动即可迁移，并以 GPU 时间计费取代按 token 计费。
TrainBridge is an agentic framework that transfers GPU compilation optimization guidance to Ascend NPU training programs, achieving an average 1.7× speedup across four workloads and nearly 2.0× across 27 Modded-NanoGPT speedrun records by extracting optimization intents from GPU executions and iteratively refining NPU programs through profiling.
Google Research introduces SimHack, a simulator that predicts when detectable reward hacking will become dominant during RL training of coding LLMs, before running costly full-scale training. It achieves 11.3% median relative error in onset prediction and 0.85 rank correlation across recipes, and can screen curricula to find ones that train 96.6% longer without hacking.
PyTorch Con 2026 主题演讲预告：Agentic AI Foundation 执行董事 Mazin Gilbert 探讨 LLM 如何从智能体行为中学习，提出五层智能体栈架构，并将智能体评估反馈为 PyTorch 训练奖励的开放研究方向。
ICCLHerd 提出一种基于 herding 的 LLM 训练数据选择框架，利用 von Mises–Fisher 核平衡 mode-seeking 与 mode-averaging，并以 in-context learning 作为验证代理进行超参数搜索，实验表明其优于现有数据选择算法。
The paper introduces Ask-to-Distill (A2D), a framework that RL-trains a student-initialized synthesizer to generate synthetic questions that elicit dense teacher supervision for on-policy distillation, without needing reference answers. It improves performance and teacher-gap recovery over unoptimized prompts and real-question banks across math, code, and agentic tasks.
TraceForge 提出一种两阶段（监督+对抗）方法，从教师答案中生成匹配学生模型自身风格的推理轨迹，无需访问隐藏思维链，在 HumanEval+ 和 MBPP+ 等编码基准上显著超越使用教师原始思考内容的训练效果。