generalization

The article proposes that Transformers can generalize to new tasks through a well-designed harness that induces composition, without needing intrinsic model generalization. It shows RLMs can generalize from short tasks to 8-32x longer tasks and across domains.

This paper introduces Bifocal Attention, which decouples positional encoding into geometric (standard RoPE) and spectral (learnable harmonic operators) modalities to address the 'Spectral Rigidity' of fixed RoPE, improving algorithmic generalization beyond the training window.

Researchers present AIDE², a system with recursive auto-research loops that improved its own code over 100 iterations, discovering seven improvements and beating a hand-tuned agent on held-out benchmarks.

Anchor-Align augments behavioral cloning with vision-language anchoring to preserve pretrained representations and language-action alignment, improving real-robot success rates by over 20% on xArm7 and showing consistent gains in simulation benchmarks.

Introduces Contrastive Weak-to-Strong Generalization (ConG), a framework that uses contrastive decoding to generate higher-quality samples from weak models for more reliable weak-to-strong generalization in LLMs, demonstrating consistent improvements across model families.

The author shares observations from auto-research experiments in algebra, noting that AI models can generate code and discover novel abstract rules, leading to potentially alien mathematics that humans struggle to understand.

This paper introduces analytically tractable Bayesian information restricted diffusion (BIRD) models to study the memorization-generalization phase transition in diffusion models, finding that generation proceeds near the edge of memorization and that information restriction helps circumvent the curse of dimensionality.

This paper systematically evaluates time series foundation models (TSFMs) for forecasting extreme PM2.5 concentrations from wildfire smoke using a 12-year dataset from California. Results show that fully-trained recurrent baselines like BiLSTM outperform TSFMs, challenging the assumption that larger pretrained models dominate in environmental forecasting.

The paper formalizes the "Knowing-Using Gap" in LLM fine-tuning, where models memorize facts but fail to generalize them. It introduces self-patching intervention and identifies knowledge-circuit misalignment as the root cause, recovering 58-75% of generalization failure with a simple heuristic.

This paper introduces GeoSD, a geometric self-distillation objective that uses Hellinger loss and a proximal Fisher-Rao distance term to counter drift in on-policy self-distillation, improving out-of-distribution reasoning accuracy by 5.7–8.6 points across model scales.

Proposes EISAM, a new optimizer that extends Sharpness-Aware Minimization using an extragradient step to find flatter minima, improving generalization and robustness while reducing sensitivity to hyperparameters. Outperforms SGD, Adam, and SAM on benchmarks.

Introduces RMISC, a large-scale real-world multivariate time series corpus with around 200 datasets and 142 billion time points, and demonstrates that pretraining time series foundation models on real-world multivariate data improves zero-shot generalization compared to synthetic data.

Direct-OPD distills the policy shift from a small model's pre- and post-RL checkpoints to improve a larger student model via on-policy distillation, achieving significant gains without expensive RL on the student.

A single AI policy has been developed that can control 20 different robot bodies, ranging from single arms to full humanoids, operating fully autonomously.

This paper introduces RiskAverseOOD, a benchmark for measuring how well risk aversion learned in low-stakes gambles generalizes to astronomically high-stakes gambles in language models. Initial results show that models like Qwen3-8B can generalize risk aversion partially across 98 orders of magnitude, though not yet reliably enough for a safety failsafe.

This paper presented at ICML explores how causal and statistical models can generalize to novel combinations of interacting objects, with a poster session scheduled at the conference.

Introduces SVA, a framework that decouples action generation from consequence evaluation in frozen VLA models using Monte-Carlo tree search and distillation into a lightweight Q-value model, improving generalization and task success rates while reducing computational costs.

This paper proposes Native Factorized Weights for transformers, where every linear layer is trained as a product of two low-rank matrices from initialization. Experiments show a corpus-determined optimal rank that minimizes validation loss and a generalization band, outperforming dense baselines with fewer parameters.

This paper observes that token embeddings in small language models condense into a narrow cone-like subspace, a phenomenon termed embedding condensation, and proposes a dispersion loss to counteract it, improving generalization.

This paper systematically studies the limitations of steering vectors for controlled text generation, finding that their effectiveness varies across traits, degrades on task transfer, and suffers from composition tradeoffs.