smoothing

A practical guide to recursive filters (SMA, EMA, low-pass, and a tiny 1D Kalman) for smoothing noisy measurements with low latency and compute.

This paper studies transformers trained on Markov chains and identifies that induction heads implement soft context-matching and Dirichlet-style smoothing, showing that transformers regularize in-context estimation rather than simply counting n-grams.