theory

This paper derives an exact closed-form analysis showing that feature learning in neural networks makes them more vulnerable to backdoor attacks, with feature-learning regimes requiring only a trigger strength scaling as α ∝ π^{-1/4} versus π^{-1/2} for lazy learners — theoretically explaining why backdooring large networks is surprisingly easy and why linear security audits underestimate the threat.

This paper examines the syntax and semantics of goal representations, analyzing their compositionality and role in rational behavior for cognitive science and AI.

Paul Dirac explores the deep connection between mathematics and physics, emphasizing their interdependence in scientific frameworks.

The author announces a speaking engagement at Harvard University where they will discuss world models, the Le* family, and the need for more theory and mathematics to advance JEPAs.

This paper develops a statistical-mechanical framework for analyzing learning dynamics in deep neural networks by shifting from parameter space to function space, deriving exact error dynamics and fluctuation-induced effects.

The Bit Radix Theory argues that different forms of intelligence, such as human and AI, may have fundamentally distinct cognitive approaches rather than converging, emphasizing complementary strengths. The paper promotes open critical engagement by inviting readers to test the theory using AI tools.

This paper analyzes how finite Newton-Schulz iterations in the Muon optimizer benefit nonsmooth nonconvex optimization by smoothing the polar map, providing convergence guarantees that match best-known bounds.

The paper proposes 'unit' as an explicit primitive in machine learning, where learning tasks declare persistent individuals, and supervised learning specializes to unit-conditioned response laws with tokenization.

The article speculates on the upcoming release of GLM 5.3 weights and suggests that OxAlpha may be a new GLM model variant.

The article discusses Julian Jaynes' theory that human consciousness is a recent development, emerging around 3,000 years ago due to a shift in brain integration, supported by historical and neurological evidence.

The paper establishes a theoretical connection between probabilistic Joint-Embedding Predictive Learning (JEPA) and Hidden Markov Models (HMMs), providing a state-space interpretation and introducing Markov-Chain JEPA for enhanced consistency.

The article questions whether theoretical principles still guide machine learning practices, highlighting how many once-standard theories have been challenged by empirical evidence.

A survey paper examining the expressive power of transformers as language recognizers, using concepts and methods from circuit complexity to compare them with classical models of computation.

This paper introduces low interaction rank as a unified theoretical framework for multiplicative dual-encoder networks, covering approximation, sample complexity, normalization, and identifiability, with experiments on operator learning and CLIP models.

This paper proves that a single normalized nonnegative kernel-attention head requires exponentially many features to solve a simple Min-IP task on three-token sequences, whereas dense softmax attention solves it with constant temperature and m-dimensional scores, highlighting a fundamental expressive-power gap between kernel and full attention.

This paper reviews the concept of 'perspective' in NLP, proposes a hierarchy of perspective-related concepts along a specificity axis, and demonstrates how this hierarchy can help researchers choose appropriate operationalizations.

A theory paper introducing Decoupled Descent (DD), a training method that uses approximate message passing Onsager corrections to enforce asymptotic equality between training and test error during gradient descent, potentially enabling better stopping and hyperparameter tuning.

This paper studies the joint effect of memory width and batch depth in stochastic Lipschitz bandits, characterizing the minimax pseudo-regret tradeoff up to logarithmic factors and showing that state width and update depth are not interchangeable.

This paper studies the sample complexity of policy learning under the mu-resets interaction protocol in reinforcement learning, resolving a question about the role of policy realizability and showing horizon dependence is exponential under all-policy concentrability and sqrt-exponential under pushforward concentrability.

This paper introduces a constant-aware comparison protocol for average-reward reinforcement learning regret bounds, deriving an explicit finite lower certificate for communicating MDPs and improving published coefficients.