generalization

This paper observes that token embeddings in small language models condense into a narrow cone-like subspace, a phenomenon termed embedding condensation, and proposes a dispersion loss to counteract it, improving generalization.

This paper systematically studies the limitations of steering vectors for controlled text generation, finding that their effectiveness varies across traits, degrades on task transfer, and suffers from composition tradeoffs.

This paper reveals a non-monotonic effect of privacy on generalization error in Byzantine-robust distributed learning: in high-noise (strong privacy) regimes, increasing privacy reduces generalization error, while in low-noise (weaker privacy) regimes, increasing privacy degrades generalization.

This paper introduces OpenAgent, a problem setting for tool-use agents in open-world scenarios with distributional shifts, and proposes Perturbation-Augmented Fine-Tuning to improve robustness. Experiments reveal that both SFT and RL agents degrade under environmental shifts.

Proposes GenDa, a unified framework for unsupervised reinforcement learning that addresses non-stationary skill semantics and brittle generalization via skill relabeling and a complementary information bottleneck, significantly improving data efficiency and generalizability.

This paper explores deep reinforcement learning for attitude control of spacecraft during hypersonic re-entry. It demonstrates that state-of-the-art RL and hybrid controllers can outperform traditional PID controllers with gain scheduling, especially when dynamics randomization is used to improve robustness and generalization.

This paper revisits the volume hypothesis, which posits that generalization in over-parameterized networks is mainly due to the larger volume of good-generalizing regions in weight space rather than SGD's implicit bias. Through experiments with binary networks, the authors show that the generalization advantage of gradient learning over random sampling diminishes as training data size grows, potentially resolving contradictory prior findings.

This paper proposes a linguistic-invariant spoofing detection framework that uses teacher-student adversarial learning and a variational information bottleneck to mitigate linguistic bias, achieving up to a 36.2% relative reduction in equal error rate across nine datasets.

This paper investigates how well sharpness and complexity together explain generalization in deep neural networks, introducing a Pareto-based analysis and function-oriented definitions to expand the explanatory scope.

Derives the closed-form gradient of the Wolkowicz-Styan upper bound on the loss Hessian eigenspectrum to guide neural network training toward flat minima, and introduces Hessian Spectral Range (HSR) Regularization. Numerical experiments show that HSR narrows the Hessian eigenvalue range, avoids sharp minima and saddle points, and achieves flat solutions comparable to Sharpness-Aware Minimization (SAM).

JERP introduces a method for LLM agents to jointly learn interpretable natural-language rules and update policy parameters from the same interaction trajectories, improving performance on AlfWorld and WebShop while maintaining inspectability.

This paper proposes using sparse autoencoders to detect out-of-distribution inputs for transformers, including typos and jailbreak prompts, by analyzing spurious concept activations. The method enables a mechanistically grounded fine-tuning strategy to improve LLM robustness.

This paper introduces directional sharpness, a new metric for certifying the generalization performance of machine learning models that is both efficient to compute and more reliable than existing proxies like test accuracy or traditional sharpness, even when training deviates from prescribed procedures.

This paper evaluates the robustness of multi-sensor fusion for cattle posture classification under temporal distribution shift, finding that multimodal models suffer significant performance drops and that simpler single-sensor models generalize better, highlighting shortcut learning issues.

SAM2Matting introduces a tracker-to-matting framework that enhances foundational trackers like SAM2 with region-proposal bridges and matting heads, achieving state-of-the-art video matting performance even when trained only on images, with strong generalization across diverse scenarios.

GauntletBench is a new web-based benchmark that evaluates AI agents on challenging scenarios focusing on temporal perception, graphical understanding, and 3D reasoning. Results show state-of-the-art agents achieve only 19.1% success rate compared to over 80% for non-expert humans, highlighting significant limitations in current agentic systems.

This paper from OpenAI investigates whether reinforcement learning on beneficial behavior can produce broad and persistent alignment generalization beyond the training distribution. Using a dataset of realistic situations, they show that RL training on beneficial traits improves out-of-distribution alignment and persistence against adversarial attacks.

This paper proposes a hierarchical multi-agent reinforcement learning framework that enforces hard safety constraints via a constraint manifold at the low level while enabling effective coordination through high-level policy learning, providing theoretical safety guarantees and achieving near-perfect safety rates with good generalization.

This paper introduces Tailor-Bench, a benchmark that systematically evaluates visual world models on irregular physical interactions, revealing a long-tail gap in generalization where models perform well on common scenarios but degrade on unconventional and impossible ones.

This paper proposes eCNNTO, a CNN with residual connections to accelerate density-based topology optimization by predicting near-optimal densities from early iteration histories, achieving up to 97% reduction in iterations and strong generalization across different boundary conditions, geometries, and mesh resolutions.