Graph spectral analysis (Fiedler value + Scheffer CSD indicators) predicts grokking 21k steps before loss function - five reproducible experiments [R]
Summary
Applies graph spectral analysis (Fiedler value) and Scheffer critical slowing down indicators to predict grokking in neural networks, detecting it 21,000 steps before the loss function changes, across five reproducible experiments.
Similar Articles
Feature Repulsion and Spectral Lock-in: An Empirical Study of Two-Layer Network Grokking
This empirical study validates theoretical findings on feature repulsion and spectral lock-in during the grokking phenomenon in two-layer neural networks, demonstrating how activation functions influence the transition from memorization to generalization.
Distributional Spectral Diagnostics for Localizing Grokking Transitions
This paper proposes distributional spectral diagnostics to localize grokking transitions in Transformer models before test accuracy rises. It uses empirical distributions and Hankel dynamic mode decomposition to create a monitoring signal that discriminates between grokking and non-grokking runs.
Noise-Driven Escape from Metastable Phases explains Grokking in Deep Neural Networks
The paper proposes that grokking in deep neural networks arises from noise-driven escape from metastable phases in first-order L2 phase transitions, demonstrating that delayed generalization follows Arrhenius scaling and reproduces canonical grokking curves.
At-Grok Is Not Converged:A Measurement-Validity Audit for Grokking Representation Metrics
This paper audits the measurement validity of representation metrics in grokking, showing that values at the grokking transition overstate converged circuit complexity and that compression lags generalization. It provides tooling to separate onset from compression and reports negative results on generality.
Detecting Neural Network Failures through Spectral Analysis of Internal Activations
This paper identifies spectral drift in internal activations of neural networks during misclassifications and introduces Self-Detecting Neural Networks (SDNN) that monitors spectral dynamics to detect failures, achieving 79% AUROC on CIFAR-10, outperforming confidence-based methods by 25-30 percentage points.