regularization

The paper shows that sparsity-inducing regularization causes functional modularity to emerge in multitask neural networks, identifying compact task-selective subnetworks ('functional tokens') that can be recombined across pathways without retraining.

本研究在受控因子实验（720次运行）中比较了门控初始化（常规 vs. chrono）与循环dropout/zoneout在LSTM和GRU上学习长程依赖的效果，发现chrono初始化在合成任务上显著优于常规初始化，而dropout通常延迟学习、zoneout仅在部分条件下有益。

该论文研究高维场景下的持续线性回归，推导了梯度下降有限步数训练时泛化损失的闭式表达，并证明最优步长按 lnT/T 衰减，其效果渐近等价于最优 L2 正则化。研究还探讨了对角正则化条件以及任务变异性与隐式正则化之间的权衡，并通过实验验证了理论预测。

该论文研究了On-Policy Self-Distillation（OPSD）在LLM自我改进过程中导致已有能力退化的问题，发现这种退化与隐藏表示几何结构恶化（簇内紧凑性和簇间分离度下降）相关，并提出表示几何正则化（RGR）方法，在不改变原有优化流程的情况下有效缓解能力退化。

该论文研究在数据有限但算力充裕场景下的重复数据预训练，发现小批量通过梯度噪声提供隐式正则化，并在固定时间预算下小批量+少GPU可取得更低验证损失，挑战了常见地增大数据并行度以最大化GPU利用率的做法。

This CIKM 2026 paper characterizes "convergence collapse" in fixed-target tabular anomaly detectors, where better optimization removes anomaly signal, and proposes Kernel Contraction Matching (KCM), a closed-form kernel-regression detector, plus the Kernel-Anchored Regularizer (KAR) to mitigate collapse and improve AUROC on ADBench.

The paper introduces TextReg, a regularization framework for text-space prompt optimization that mitigates prompt distributional overfitting via dual-evidence gradient purification and semantic edit regularization, yielding up to +11.8% over TextGrad and +16.5% over REVOLVE on out-of-distribution reasoning benchmarks.

This paper develops a statistical theory of grokking in high-dimensional regression, showing how regularization geometry and signal sparsity govern generalization near interpolation, including a zero-one generalization law and instability of minimum-norm interpolators.

SkillEvoReg introduces a regularization framework to prevent overfitting in the skill evolution of language-model agents, combining dropout, local regularization, and causal validation to maintain performance while controlling skill growth.

FuseReg is a regularization method for Representation Autoencoders that mitigates the reconstruction-generation gap by using random layer-subset sampling during training, improving generation performance and decoder robustness across different layer inputs.

The paper proposes a spectral connectivity-regularized graph learning framework (SCoGL) that incorporates Laplacian spectral priors to improve graph recovery and downstream tasks like graph signal denoising when data is scarce.

SPAR is a pretraining objective that uses a gated KL loss to stabilize language model predictions against irrelevant prefix text, improving robustness in long-context scenarios as demonstrated on multiple benchmarks.

This paper introduces Regularized Recursive Self-Improvement (RRSI) for AI agent harnesses, which applies regularization to prevent overfitting during recursive evolution, demonstrating performance gains on multiple benchmarks.

The paper explores grokking, a delayed generalization phenomenon in neural networks, and introduces Geometric Dimensionality Regularization (GeomDR) to control representation geometry, accelerating grokking by up to 52x.

The paper introduces Regularized Emphatic Temporal-Difference Learning (RETD), which ensures stability under constant stepsizes by normalizing the emphatic TD signal, with proofs of convergence and experimental validation.

This paper investigates the limits of proper learning and regularization in multiclass learning, resolving open problems by demonstrating that learning cannot always be reduced to proper learning and that regularization has structural constraints.

This paper proposes a training-time explainability framework for multilingual hate speech detection, aligning model reasoning with human rationales to improve classification performance and interpretability, evaluated on English and Hinglish datasets.

The paper introduces ERPO, a method that moves regularization from the action-side to the input-side by controlling query distribution, addressing the stability-exploration dilemma in LLM policy optimization, and showing improvements on mathematical reasoning benchmarks.

This paper presents a compositional theory of curvature in probabilistic circuits, showing that the Hessian trace factorizes per sum node into circuit flow and local sharpness, and introduces an adaptive sharpness-aware regularizer that preserves closed-form EM updates while improving generalization.

A paper introduces TC-LeWM, applying regularization losses like SIGReg and VISReg to temporal latent residuals to enable multi-task learning in LeWM, significantly improving success rates on the LIBERO robot arm benchmark.