Tag
A paper from UIUC, CMU, Rutgers, Princeton, Stanford, and Yale shows a 27B open language model can rewrite its own weights through reasoning alone—no gradients or training—improving accuracy by 6.6 points on unseen problems, repairing a damaged copy of itself, with the ability emerging only at scale, suggesting a step toward recursive self-improvement.
An experimental language model architecture uses hypersurfaces for dynamic weight updating to reduce training parameters, achieving better performance than unrolled baselines while using only 16% of the parameters. The approach is tested on the FineWeb-Edu dataset and includes a GitHub implementation.