riemannian-metric

#riemannian-metric

Your transformer's attention entropy collapse isn't a bug. It's the model doing exactly what you trained it to do. Here's how to fix it with a three-line temperature schedule. arXiv-able. Self-contained proof. No citations needed.

Reddit r/ArtificialInteligence ↗ · 2026-06-02

The article explains that attention entropy collapse in deep transformer layers is a geometric consequence of training, not a bug, and proposes a three-line temperature schedule to prevent it.

0 favorites 0 likes

#riemannian-metric

Geometric Kolmogorov--Arnold Network (GeoKAN)

arXiv cs.LG ↗ · 2026-05-11 Cached

This paper introduces Geometric Kolmogorov-Arnold Networks (GeoKAN), a family of geometry-aware models that learn Riemannian metrics to adapt coordinates for improved function approximation and physics-informed learning.

0 favorites 0 likes

riemannian-metric

Your transformer's attention entropy collapse isn't a bug. It's the model doing exactly what you trained it to do. Here's how to fix it with a three-line temperature schedule. arXiv-able. Self-contained proof. No citations needed.

Geometric Kolmogorov--Arnold Network (GeoKAN)

Submit Feedback