Tag
This paper presents a descent-free and alignment-free method to measure singular structure in trained neural networks. It recovers the order of dead directions from the directional Fisher rate, classifying genuine singularities from flat gauge symmetries, and demonstrates the technique on transformer and convolutional layers.