linear-self-attention

This paper constructs a transformer with linear self-attention that performs in-context learning of the closed-form least squares solution for simple linear regression, using layer normalization to approximate the analytical solution rather than gradient descent.