generalization

A new paper reveals "mode-hopping" in LLM pre-training, where models repeatedly toggle between shallow pattern-matching and genuine generalization even as training loss stays flat — suggesting more pre-training does not always guarantee better generalization.

This paper shows that being a valid curvature upper bound does not automatically justify using a last-layer relative-flatness proxy as an adversarial-robustness certificate or as a differentiable training regularizer. The authors derive a gauge-invariant repair, demonstrate the proxy's unboundedness under symmetry-preserving softmax shifts, and show via CIFAR-10 experiments that quotient-regularized predictors stay aligned while raw-regularized ones can have their generalization suppressed.

MedKIT is a new benchmark evaluating how well LLMs integrate and apply updated clinical knowledge across lexical, relational, compositional, and operational probes. A large-scale study of 12 knowledge integration strategies across 5 models reveals a persistent gap between factual recall and usable knowledge, with little improvement on compositional or operational reasoning.

该论文提出 Stability-Aware Generalization Objective（SAGO）框架，主张 LLM 的泛化应评估为稳定性而非准确率，通过生成一致性、内部激活、置信度和响应镜像等多维度变异测量揭示了现有模型普遍存在且与排名相关的泛化不稳定问题。

该论文提出"模式跳变"(mode-hopping)概念，指出LM预训练中平滑下降的损失与稳定基准分数会掩盖模型答题方式的突变（如OLMo3-32B在算术提示上81%→0%→81.7%的反复），并通过六项廉价行为测试排除了评估指标非线性与局部优化波动等简单解释。

The paper introduces OPSFT (On-Policy direction-constrained Supervised Fine-Tuning), which constrains SFT parameter update directions to those observed in on-policy RL training, showing that on-policy generalization advantages can be transferred to SFT for improved efficiency and the ability to leverage high-quality trajectories.

This paper finds that latent world action models fail to generalize and proposes Simple-WAM, which improves generalization performance while maintaining efficiency by simplifying future modeling into a single forward pass.

The article discusses whether AI critics Ilya Sutskever and Yann LeCun were wrong about LLMs not reaching human intelligence, as there is emerging evidence of LLMs generalizing across domains.

The paper proposes a lightweight, rule-based selector for diverse SFT traces to improve post-RL generalization in reasoning models, showing significant performance gains on mathematical benchmarks.

The paper demonstrates that residual scale alone is insufficient for learnability in delta-learning, and introduces a diagnostic metric and target design principles to improve generalization in scientific machine learning.

This paper introduces a generalized multimodal foundation model capable of handling arbitrary modality combinations and prediction tasks, achieving competitive performance through training on large-scale synthetic datasets with diverse causal structures.

A critique of Figure's Helix 2.5 robot, challenging its zero-shot generalization claims by pointing out high failure rates in trials and limited real-world applicability in demos.

Figure.ai released a 4-hour video demonstrating their Helix 2.5 humanoid robot working in 30 new homes without any additional training, showcasing its generalization capabilities.

The paper explores grokking, a delayed generalization phenomenon in neural networks, and introduces Geometric Dimensionality Regularization (GeomDR) to control representation geometry, accelerating grokking by up to 52x.

Figure releases Helix 2.5, an AI model that enables robots to generalize household tasks zero-shot across 30 homes without additional training.

The paper introduces Sharp Mode Connectivity (SMC) to enhance the robustness of weight-space interpolation in neural networks by optimizing for flatness along entire paths, achieving significant accuracy improvements under distribution shifts.

This paper uses Transition Games to study the transition from memorization to generalization in Transformers, revealing distributed utility gain with attention bias and Fourier recoding rather than a module switch.

The article explains why machine learning research agents avoid overfitting during iterative benchmark evaluations, attributing the phenomenon to compression and the role of few-token fits in generalization.

This paper assesses the generalization of nnU-Net for brain tumor segmentation in the BraTS-GoAT 2026 challenge, reporting performance metrics and analyzing failure cases.

This paper quantifies the memorization-to-generalization transition (grokking) in neural networks through scaling laws, revealing that data complexity is the primary driver of transition time compared to model capacity.