regularization

This paper identifies 'suboptimal collapse' in RL post-training of time series foundation models and proposes Ground-Truth Neighborhood Regularization (GTN-R) to keep output distributions near the ground truth, improving forecasting performance.

This paper proposes BRACE, a method that encodes an Ordered Reasoning Chain to detect ever-shifting harmful chat dialogue, achieving high harm-type F1 scores with both encoder and decoder backbones.

PhysAttNet is a physics-informed attention framework that augments lightweight CNN forecasters with domain-guided regularization to improve accuracy and generalization in industrial and astrophysical time series forecasting.

This paper proposes a neural ODE-based regularization method that enforces latent embeddings in reinforcement learning agents to follow consistent ODE flows, aligning representation learning with environment dynamics and yielding performance gains on Atari and gridworld benchmarks.

This paper proposes SCORE, a self-concordance-inspired quasi-Newton method for training physics-informed neural networks (PINNs). It uses a decrement-coupled shifted secant geometry to improve final accuracy on nonlinear PDE benchmarks without requiring Hessian computations.

This paper introduces a novel measure called relative parameter importance for task-agnostic, replay-free continual learning, enabling better balance between stability and plasticity by regularizing only parameters critical for past tasks while allowing others to update for backward knowledge transfer. The method is evaluated on class-incremental and domain-incremental text classification tasks.

This paper introduces Omega-S, a lightweight, data-free regularization penalty for low-rank fine-tuning that improves retention of original model capabilities by penalizing variance in weight-matrix node degrees. Experiments on Llama-3-8B with LoRA show it retains more original capability than no regularization or tuned baselines.

This paper introduces Modality Contribution Drift (MCD) in multimodal continual learning and proposes CMCDR, a regularization method with replay-based and replay-free variants to preserve modality contribution structures across incremental tasks.

This paper proposes QQWorld, a quantile-quantile matching objective that replaces the Epps-Pulley objective in LeWorldModel for better regularization of latent distributions, improving planning success in control environments.

Proposes Unbiased Open World Regularization (UOWReg), an encoder-only framework that enforces conditional distribution matching to achieve statistical independence between learned representations and sensitive attributes, reducing bias while maintaining accuracy.

Explains SIGReg, a novel regularizer for JEPA that prevents representation collapse by forcing embeddings to follow an isotropic Gaussian distribution, with theoretical guarantees and a clean training loop.

This paper introduces a regularization technique for Direct Alignment Algorithms (DAAs) that maintains normalized response probabilities, mitigating over-optimization and likelihood displacement. The method improves generation quality and benchmark performance, achieving over 20% relative increase on AlpacaEval2 and 9% gains on general benchmarks for Llama-3.1-8B-Instruct.

This paper introduces OrthoGrad, a geometric intervention that removes the radial component of weight gradients during optimization, and shows that it reduces memorization of noisy labels in small-data regimes but does not prevent eventual memorization.

This paper identifies that standard ridge regularization in potential recovery from flow on directed graphs collapses and reverses the ordering of the estimate due to gauge dependence. It proposes a gauge-invariant Dirichlet energy penalty that yields a parameter-insensitive solution and demonstrates robust dynamic range preservation on real clickstream data, with implications for preventing oversmoothing in graph neural networks.

Google Research shows that the creativity of diffusion models is a mathematical consequence of neural network regularization causing score smoothing and interpolation, demystifying their ability to generate novel data rather than just memorize.

This paper introduces Feedback Manipulation Regularization (FMR), an algorithm-agnostic method that uses evaluative feedback to improve alignment in imitation learning, achieving up to 98% reduction in misalignment in Safety Gymnasium environments.

This paper proposes FedFMX, a Fisher-Routed Mixture of Experts framework for Federated Class-Incremental Learning, addressing capacity conflict, catastrophic forgetting, and data heterogeneity via adaptive expert specialization.

Derives the closed-form gradient of the Wolkowicz-Styan upper bound on the loss Hessian eigenspectrum to guide neural network training toward flat minima, and introduces Hessian Spectral Range (HSR) Regularization. Numerical experiments show that HSR narrows the Hessian eigenvalue range, avoids sharp minima and saddle points, and achieves flat solutions comparable to Sharpness-Aware Minimization (SAM).

Introduces VISReg, a regularization method for JEPA (Joint Embedding Predictive Architecture) training that combines variance, invariance, and sketching constraints.

A Twitter thread listing 13 core machine learning concepts that interviewers expect candidates to know, covering topics from bias-variance tradeoff to the curse of dimensionality.