feature-learning

This thesis elucidates mode connectivity in neural network loss landscapes through spectral dynamics, explaining phenomena like grokking and memorization, and applies insights to enhance diverse feature learning and practical neural network training.

The paper shows that multilayer perceptrons naturally develop monosemantic specialized neurons that improve data efficiency by learning local low-dimensional representations instead of a global one in regression problems with clustered data.

SUFLECA is a weakly-supervised framework for zero-shot CAD-to-image alignment, achieving state-of-the-art accuracy on ScanNet25k by scaling up geometry-grounded feature learning from pretrained visual representations.

This paper introduces ODIN, a novel autoencoder architecture that enforces orthogonality and importance ordering of latent dimensions, recovering PCA-like interpretability in a fully non-linear regime. The method integrates geometric constraints into the training objective, theoretically grounded and empirically validated on synthetic and real-world datasets.

This paper theoretically demonstrates that two-layer neural networks trained on group composition tasks learn spectral representations, with neurons converging to irreducible representations and achieving rotational rank-one alignment, providing a representation-theoretic account of feature learning.

This paper introduces GOEN, a pipeline combining multi-scale features, L2 normalization, and Mahalanobis distance for OOD detection, and finds that CenterLoss regularization actually degrades OOD performance despite improving classification accuracy.

The paper introduces AGOP-Weighted, a post-hoc attribution method that multiplies per-sample gradients by a training-distribution prior to suppress noise and highlight important pixels, and demonstrates significant improvements over existing methods on synthetic and photorealistic benchmarks.

This empirical study validates theoretical findings on feature repulsion and spectral lock-in during the grokking phenomenon in two-layer neural networks, demonstrating how activation functions influence the transition from memorization to generalization.