generalization

Proposes the Piggyback Hypothesis that chat-template tokens can cause emergent misalignment in LLMs, and introduces Token-Regularized Finetuning (TReFT) to mitigate it while preserving in-domain learning.

Experts warn that viral humanoid robot demonstrations often mislead the public and investors, as robots shown performing impressive feats typically cannot generalize those skills across varied real-world conditions. Researchers from Agility Robotics and Physical Intelligence highlight the significant gap between curated demos and actual robot capabilities.

Imaginative Perception Tokens (IPT) enhance vision-language models' spatial reasoning by externalizing intermediate perceptual representations from alternative viewpoints, outperforming traditional text-based reasoning on perspective taking, path tracing, and multiview counting tasks.

Introduces delayed per-step reward attribution with eligibility gating for reinforcement learning in multi-agent language model interactions, achieving first place in the MindGames Arena benchmark at NeurIPS 2025.

This paper theoretically studies how transformer-based policies acquire search capabilities from reinforcement learning training dynamics in a stochastic tree environment. It shows that a two-head transformer can implement depth-first search and that this mechanism emerges naturally from sparse reward signals under a depth-wise curriculum.

This paper investigates whether auto-generated labels for sparse autoencoder features generalize across languages and scripts, using Serbian digraphia as a controlled testbed. It finds that while feature sets show substantial overlap across languages, the labels often fail to track the same concept in non-English inputs, particularly in less represented scripts.

GraphARC is a new benchmark for abstract reasoning on graph-structured data, extending the ARC paradigm to graphs. Evaluations of state-of-the-art language models reveal a comprehension-execution gap and performance degradation on larger instances, highlighting scaling challenges.

This paper studies representation collapse in sequential post-training of large language models, showing that repeated adaptation stages compress internal representations, reducing plasticity and out-of-domain generalization. The authors propose lightweight interventions to preserve future learnability without sacrificing behavioral gains.

This paper introduces Semi-Supervised Noise Adaptation (SSNA), a novel framework that uses synthetic noise domains (e.g., Gaussian distributions) as surrogate source domains to improve generalization in semi-supervised learning settings. The proposed Noise Adaptation Framework (NAF) establishes a generalization bound and demonstrates improved target domain performance.

This paper introduces the concept of 'initialization memory' to study how much of the random initialization bias survives training in deep networks, showing that low-learning-rate SGD preserves initialization while Adam-family optimizers erase it, and linking this to forgetting dynamics.

This paper introduces the log-alignment ratio (LAR), a training-time metric that measures parameter-activation alignment and predicts generalization by capturing the spread of weight and activation spectra. Experiments on grokking and a 3B-parameter language model show LAR tracks the transition from memorization to generalization and flags overfitting without held-out data.

This paper investigates why CLIP struggles with concept binding, showing that while CLIP's binding function is high-complexity, controlled transformer models can learn low-complexity binding functions through multiplicative interactions that generalize better.

This paper introduces an in-vitro framework with two procedurally generated languages to study cross-lingual generalization in language models, finding that tokenization's preservation of reusable substructure is more critical than lexical similarity or data balance for transferring capabilities across languages.

This paper develops a PAC-Bayesian framework for physics-informed machine learning, providing high-probability generalization guarantees for unbounded losses. It proposes a multi-task perspective that jointly handles data fidelity, PDE residuals, and boundary conditions, and introduces a self-bounding learning algorithm.

This paper conducts a causal-inspired analysis of automated prompt optimization across frameworks, LLMs, and tasks, identifying that specific edit types (e.g., complexity-increasing, meta-instructional) have systematic negative or positive effects depending on task characteristics, explaining generalization failures.

This paper proves a finite-sample bound on the approximate max-information of DP-SGD that is at most linear in dataset size, yielding PAC-Bayes generalization bounds for models trained with differential privacy.

This paper identifies neural network training as a search through Hamilton-Jacobi initial-value problems, showing that residual networks, transformers, and RNNs discretize the same class of viscous Hamilton-Jacobi equations. It derives quantitative consequences including minimax optimal generalization rates, adversarial robustness bounds, and a closed-form influence function.

A critique arguing that training LLMs on human-generated data limits their ability to discover novel solutions via test-time compute, and that true AGI requires models that can explore hypothesis spaces more broadly, similar to AlphaZero.

This paper introduces Filtered Posterior Mean Collections (FPMCs), a unified framework for analytical models of diffusion generalization, consolidating existing approaches and demonstrating improvements through soft relaxations and source distribution augmentations.

This paper introduces ASAM (Adversarial Subspace Alignment for Multimodal Knowledge Editing), a method that improves the generality of intrinsic multimodal knowledge editing by generating adversarial yet semantically coherent variants and enforcing low-rank alignment of representations, addressing the limited generalization of existing methods.