Tag
Introduces Least Square Policy Distillation (LSPD), an RL-based method for sample-efficient distillation in large language models, improving performance on mathematical reasoning benchmarks.
The article discusses the problem of exploding variance in estimating log-sum-exp functions in machine learning and proposes using least-squares regression to mitigate statistical disadvantages.
This paper establishes a sharp Loewner envelope for coefficient covariance under volume-sampled least squares, using feature-only geometry to analyze tightness conditions.
This paper proposes Fourier Feature Networks (FENs), a single-hidden-layer neural network architecture using Fourier features to solve partial differential equations, achieving higher accuracy than Extreme Learning Machines without affine transformations.
This paper constructs a transformer with linear self-attention that performs in-context learning of the closed-form least squares solution for simple linear regression, using layer normalization to approximate the analytical solution rather than gradient descent.
Proposes a hybrid least squares/gradient descent method for MIONets to accelerate training by using alternating least squares for the last layer parameters of multiple branch networks, leveraging Kronecker and Khatri-Rao products.
This paper presents an algorithm for group distributionally robust least squares regression using block Lewis weights, achieving improved complexity over interior point methods. It also provides interpolating algorithms between average and robust losses.
This simulation study examines the double descent phenomenon for least-squares interpolation on contaminated data in linear regression, comparing the performance of the least-squares interpolator with robust alternatives.