training-stability

The paper proposes Calibrated Clipping to stabilize FP8 quantization in reinforcement learning for LLMs by aligning clipping bounds with high-precision distributions, eliminating entropy surges and restoring performance.

This paper analyzes the backward-pass geometry of Z-loss in AI model training, providing a framework for understanding and optimizing gradients in dense output heads and sparse mixture-of-experts routers, with evaluations on GPT-2 and Pythia models.

The paper presents Qwen3.8-Flash-Next, a sparse mixture-of-experts AI model that enhances efficiency, capability, and training stability through architectural innovations like hybrid attention and n-gram embeddings.

Normalized Low-Rank Adaptation (NoRA) stabilizes LoRA training by normalizing down-projection matrices, accelerating convergence and improving performance without extra parameters or inference cost.

This paper investigates whether pretraining LLMs on artificial languages (pre-pretraining) consistently improves token efficiency across multiple natural languages, finding that gains are highly dependent on experimental setup and random seed, though stable gains appear for small models with the Llama tokenizer.

S2T-RLHF proposes a sentence-to-token reward decomposition framework that improves training stability and robustness in preference-based RLHF by assigning sequence-level rewards at sentence granularity, avoiding the instability of overly fine-grained token-level refinement.

This paper proposes QUADS, a method to stabilize NVFP4 reinforcement learning for mixture-of-experts large language models by aligning quantization errors between trainer and rollout sides, achieving BF16-level accuracy and higher throughput than FP8.

The paper introduces Single-rollout Asynchronous Optimization (SAO) to address stability and off-policy challenges in asynchronous RL for LLM post-training, and demonstrates that SAO consistently outperforms GRPO on agentic coding and reasoning benchmarks.

Tsinghua's SAO algorithm addresses stability and off-policy drift in asynchronous reinforcement learning for LLMs, achieving consistent improvements over GRPO on agentic coding and reasoning benchmarks, and is used to train the GLM-5.2 model.

Trust Region Policy Distillation (TOP-D) stabilizes on-policy distillation by dynamically constructing a proximal teacher, providing theoretical convergence guarantees and empirical gains in mathematical reasoning tasks with zero additional overhead.

This paper introduces FADE (Focal Advantage with Dynamic Entropy), a self-adapting advantage function that dynamically schedules gradient weights during RL post-training of LLMs, achieving faster convergence and better accuracy-diversity trade-offs compared to static baselines.

A blog post summarizing ten recent agentic RL frameworks and best practices, covering modular interfaces, trajectory structure, action masks, process rewards, advantage normalization, scalable rollouts, stability/exploration, and task curriculum.

This paper analyzes token-level gradient dynamics in RLVR training, revealing how advantage sign and token probability jointly affect update stability, and introduces Winner Advantage Policy Optimization (WAPO) which performs clipped updates only on positive-advantage completions to improve stability.

AAD-1 introduces asymmetric adversarial distillation with phased training to achieve one-step autoregressive video generation, outperforming prior methods on VBench.

Fern announces a new regularization technique that solves the SolidGoldMagikarp stability problem, with details to follow in the thread.

DVAO adaptively weights objectives based on reward variance to improve multi-reward RL training stability and multi-objective performance.

This paper identifies gradient oscillation and residual explosion as causes of training instability in Looped Transformers, and proposes Fully Looped Transformer with two parameter-free modifications (Fully Looped Architecture and Attention Injection) to stabilize training up to 12 loop iterations, achieving up to 13.2% improvement in downstream performance.

This paper introduces LBW-Guard, a bounded autonomous training control governance layer that operates above the AdamW optimizer to monitor telemetry and apply bounded control during training, demonstrating improved perplexity and training speed under stress conditions.

Introduces Implicit Behavior Policy Optimization (IBPO), a counterfactual comparison-based credit assignment framework that improves training stability and performance in multi-step reasoning tasks for large language models by converting sparse terminal rewards into step-sensitive learning signals.

The author announces the release of their first AI research paper, STAM (Stable Training with Adaptive Momentum), a new deep learning optimizer addressing stability and resource efficiency, and invites feedback from the AI community.