kernel-ridge-regression

Tag

Cards List
#kernel-ridge-regression

Kernel Ridge Attention: Nonlinear Memory with Selective Forgetting

arXiv cs.LG ↗ · 2d ago Cached

Kernel Ridge Attention (KRA) is a recurrent sequence mixer that uses recursive ridge regression with covariance-guided memory updates and an Ornstein-Uhlenbeck transition for selective forgetting, achieving constant-size inference state and improved language-modeling performance over attention and recurrent baselines.

0 favorites 0 likes
#kernel-ridge-regression

A mathematical theory of balancing relational generalization and memorization

arXiv cs.LG ↗ · 2026-05-25 Cached

This paper introduces a novel task, transitive inference with exceptions, and analytically characterizes how neural network models (kernel ridge regression) balance relational generalization and memorization. The theory is validated in pretrained language models, showing systematic mistakes predicted by the theory.

0 favorites 0 likes
← Back to home

Submit Feedback