Tag
This paper investigates how class label encoding influences neural collapse in neural network classifiers, showing that with one-hot encoding and balanced data, uncentered mean features transition from a simplex equiangular tight frame to an orthogonal frame as bias regularization increases.
This paper studies how depth alone induces an implicit low-rank bias in deep unconstrained feature models trained without regularization, shifting the optimal solution from neural collapse to softmax codes, and provides the first asymptotic and dynamic characterization of this bias under gradient descent with cross-entropy loss.
This paper shows that cross-entropy and supervised contrastive learning are both forms of prototype learning on the hypersphere and proposes normalized losses (NTCE and NONL) that achieve Neural Collapse by design, outperforming standard methods.