theory

This paper proposes a realizability theory for length generalization in a controlled single-head attention setting, deriving via a simulation transfer operator why relative and absolute positional encodings differ in extrapolation behavior, with experiments on 2625 trained models confirming the structure.

该论文研究随机舍入（stochastic rounding）如何影响低精度训练中的预测风险，针对两层线性网络推导了匹配的上下界，并给出了随计算量、模型规模、训练时长与精度变化的 scaling laws，揭示前向与后向舍入对最优计算缩放的不同影响。

该论文从随机最优控制的角度为 Transformer 在非马尔可夫数据上的近无损下一词预测提供了数学理论，构造了深度 M+1 的 Transformer 实现截断滤波器，并在平稳遍历性或滤波稳定性条件下证明误差可随上下文长度和数据量任意减小。

The paper characterizes when unlabeled data pools can substitute for membership queries in learning halfspaces, showing near-optimal label complexity under mild angular assumptions on the marginal and separating results when those assumptions fail.

A theory paper showing that in RLVR, every pass@1 improvement provably costs performance on some prompt under strategy specialization, with a tight bound on the number of pass@k crossover points, framed as an optimal tradeoff frontier rather than a bug.

该论文定义了一种与 Fisher 信息矩阵对齐的参数分布，使得在保证低损失的同时获得与 Fisher 信息矩阵零特征值数量成正比的参数对数体积，从而得到一个随 Fisher 信息矩阵期望秩缩放的高概率 PAC 泛化误差界。

The paper proves that the closed-form bounds proposed by Shu et al. for probabilities of causation are sharp in every dimension, and shows that PNS, PSub, PRep, and PN are all special cases of a single unified theorem for discrete PoCs.

该论文研究中心化二次神经网络的谱可辨识性与学习曲线辨识问题，构造了谱摘要一致但损失曲线不同的输入分布，并在高斯输入下证明单条精确标量损失曲线可辨识初始预测器与教师谱。

Concordia University and Mila researchers derive exact scaling laws for prediction regret in a bottlenecked bigram model with Zipfian token statistics, isolating capacity-driven scaling from optimization effects and matching predictions on WikiText2 and OpenWebText.

该论文提出一个群体级别的尺寸一致逼近框架，证明带连续随机特征的 MPNN 在跨图尺寸时并非通用，并分析了随机着色 MPNN 与高阶 Weisfeiler-Leman 方法之间的不可比较性。

该论文引入了一种以神经网络复杂度（Barron能量）为核心的最优传输框架，替代传统L²动能，从而在概率测度空间上定义新的几何结构，并证明了扩散生成模型在Barron几何下的次优性（超多项式score逼近下界）。

该论文研究在用户偏好存在异质性时，通过成对比较按总体平均效用对模型进行排序所需的重复比较次数。作者提出了一种 Russian Roulette 式的 MLE 算法，仅需每上下文 O(log 1/Δ) 次（期望 O(1) 次）重复比较即可恢复排序，并证明了该对数依赖是最优的。

This arXiv paper introduces the Language Model Agent Machine (LAM), a resource-bounded framework that quantifies the computational costs of LLM agent harnesses (context, memory, tools, verification), establishing lower bounds and reliability results validated on GPT-6 Astra.

The paper provides theoretical guarantees for beam-search-based test-time scaling in LLM reasoning, proving a lower bound on vanilla beam search sample complexity and proposing confidence-filtered beam search (CF-Beam) that reduces coverage dependence from quadratic to nearly linear, outperforming sequence-level methods like Best-of-N on long-horizon tasks.

This UCLA paper theoretically analyzes on-policy distillation (OPD) versus off-policy/SFT-style distillation, showing how forward and reverse KL objectives aggregate multiple teachers into arithmetic vs geometric mixtures, establishing logarithmic regret bounds, and explaining both the benefits and fragility of OPD (e.g., sensitivity to low-probability teachers and incorrect-prefix bias).

This Texas A&M paper introduces "semantic recoverability" to explain which downstream tasks survive same-instance self-supervised learning, linking recoverability to class-distance-normalized variance, few-shot nearest-centroid transfer, and the spectral subspace spanned by cross-view-stable modes of the SSL objective.

This arXiv preprint develops a theory of context sufficiency for generative AI personalization, proposing a Context-Sufficiency Frontier that distinguishes four states (insufficiency, sufficiency, saturation, interference) and showing via a full-factorial experiment with a generative recommender at a home-furnishing retailer that relevant context improves appropriateness while irrelevant context harms it and destabilizes retrieval.

iADD proposes an incremental Feynman-Kac training method for diffusion policy optimization that provides better alignment-diversity tradeoffs, and theoretically shows that only late-timestep updates in reverse diffusion may harm diversity, contrary to prior work.

This paper derives an exact closed-form analysis showing that feature learning in neural networks makes them more vulnerable to backdoor attacks, with feature-learning regimes requiring only a trigger strength scaling as α ∝ π^{-1/4} versus π^{-1/2} for lazy learners — theoretically explaining why backdooring large networks is surprisingly easy and why linear security audits underestimate the threat.

This paper examines the syntax and semantics of goal representations, analyzing their compositionality and role in rational behavior for cognitive science and AI.