feature-learning

The paper introduces Shaped SAEs, including a Matched Filter SAE (MF-SAE), which linearly shapes LLM activations via Jacobian-based rank-reducing transformations before learning sparse autoencoders to improve semantic selectivity of dictionary atoms, evaluated on SAEBench and SAE Probes.

This paper explores adapting prior-data fitted networks (PFNs) for tabular anomaly detection, proposing a fine-tuning-free approach (ZEN) using frozen TabPFN features with nearest-neighbor scoring, and a fine-tuned method (FOCUS) that surpasses all baselines on the ADBench benchmark.

This paper derives an exact closed-form analysis showing that feature learning in neural networks makes them more vulnerable to backdoor attacks, with feature-learning regimes requiring only a trigger strength scaling as α ∝ π^{-1/4} versus π^{-1/2} for lazy learners — theoretically explaining why backdooring large networks is surprisingly easy and why linear security audits underestimate the threat.

This thesis elucidates mode connectivity in neural network loss landscapes through spectral dynamics, explaining phenomena like grokking and memorization, and applies insights to enhance diverse feature learning and practical neural network training.

The paper shows that multilayer perceptrons naturally develop monosemantic specialized neurons that improve data efficiency by learning local low-dimensional representations instead of a global one in regression problems with clustered data.

SUFLECA is a weakly-supervised framework for zero-shot CAD-to-image alignment, achieving state-of-the-art accuracy on ScanNet25k by scaling up geometry-grounded feature learning from pretrained visual representations.

This paper introduces ODIN, a novel autoencoder architecture that enforces orthogonality and importance ordering of latent dimensions, recovering PCA-like interpretability in a fully non-linear regime. The method integrates geometric constraints into the training objective, theoretically grounded and empirically validated on synthetic and real-world datasets.

This paper theoretically demonstrates that two-layer neural networks trained on group composition tasks learn spectral representations, with neurons converging to irreducible representations and achieving rotational rank-one alignment, providing a representation-theoretic account of feature learning.

This paper introduces GOEN, a pipeline combining multi-scale features, L2 normalization, and Mahalanobis distance for OOD detection, and finds that CenterLoss regularization actually degrades OOD performance despite improving classification accuracy.

The paper introduces AGOP-Weighted, a post-hoc attribution method that multiplies per-sample gradients by a training-distribution prior to suppress noise and highlight important pixels, and demonstrates significant improvements over existing methods on synthetic and photorealistic benchmarks.

This empirical study validates theoretical findings on feature repulsion and spectral lock-in during the grokking phenomenon in two-layer neural networks, demonstrating how activation functions influence the transition from memorization to generalization.