theoretical

An illustrative model explaining how a 4x operating gap can arise under identical model and usage constraints, highlighting efficiency divergence mechanisms.

This paper proposes a theoretical framework for intelligence as atomic compression and compositional reuse, introducing the Compression Calculus and the Compounding Cascade thesis.

This paper quantifies and expands the theoretical capacity of late-interaction retrieval models, showing that MaxSim can replicate inner products between non-negative vectors and proposing Signed MaxSim for arbitrary real-valued vectors, revealing a representation gap between inner product and late-interaction models.

Derives the closed-form gradient of the Wolkowicz-Styan upper bound on the loss Hessian eigenspectrum to guide neural network training toward flat minima, and introduces Hessian Spectral Range (HSR) Regularization. Numerical experiments show that HSR narrows the Hessian eigenvalue range, avoids sharp minima and saddle points, and achieves flat solutions comparable to Sharpness-Aware Minimization (SAM).

An exploration of AI agents conceptualized as webs of beliefs, discussing implications for AI alignment and understanding agency.

This book presents a mathematical theory of deep representation learning, aiming to demystify the internal mechanisms of large deep networks using optimization and information theory, making architecture design a matter of linear algebra and calculus.

This paper proves sharp dimension-free first-order lower bounds for finding epsilon-stationary points in higher-order smooth nonconvex optimization, resolving open problems for Hessian-Lipschitz and third-order smooth cases.

This paper revisits the theoretical foundations for detecting commutative factors in factor graphs, correcting a previously mistaken sufficient condition and presenting corrected algorithms.

This paper identifies a collapse-and-refine mechanism in diffusion models under the manifold hypothesis, proposing Score-induced Latent Diffusion (SiLD) that provably avoids the curse of dimensionality. Experiments show SiLD matches or outperforms VAE-based latent diffusion models.