Neural Networks Provably Learn Spectral Representations for Group Composition

arXiv cs.LG Papers
neural-networks group-theory representation-learning spectral-methods deep-learning-theory feature-learning

Summary

This paper theoretically demonstrates that two-layer neural networks trained on group composition tasks learn spectral representations, with neurons converging to irreducible representations and achieving rotational rank-one alignment, providing a representation-theoretic account of feature learning.

arXiv:2606.02993v1 Announce Type: new Abstract: Understanding how structured internal structure emerges during neural network training is central to the study of deep learning. We investigate this phenomenon through the group composition task, where a two-layer neural network is trained to predict $g_1 \star g_2$ for elements of a finite group $G$. By lifting the projected gradient flow to the Fourier domain, we demonstrate that the training dynamics are governed by a Riemannian gradient ascent on a representation-theoretic energy functional. We prove that, under random initialization, this flow drives each neuron to converge almost surely toward a single irreducible representation, while the cross-layer Fourier coefficients achieve a rotational rank-one alignment. This framework provides a representation-theoretic account of feature learning and characterizes a novel low-rank compression phenomenon for matrix-valued group representations. Moreover, for Abelian groups, we provide a complete population-level description: random initialization promotes uniform diversification across nontrivial representations and induces Haar-uniform phases, jointly approximating the indicator via a majority-vote mechanism. We further prove that both phase alignment and representation competition emerge with exponential convergence rates.
Original Article
View Cached Full Text

Cached at: 06/03/26, 09:41 AM 

# Neural Networks Provably Learn Spectral Representations for Group Composition
Source: [https://arxiv.org/abs/2606.02993](https://arxiv.org/abs/2606.02993)
[View PDF](https://arxiv.org/pdf/2606.02993)

> Abstract:Understanding how structured internal structure emerges during neural network training is central to the study of deep learning\. We investigate this phenomenon through the group composition task, where a two\-layer neural network is trained to predict $g\_1 \\star g\_2$ for elements of a finite group $G$\. By lifting the projected gradient flow to the Fourier domain, we demonstrate that the training dynamics are governed by a Riemannian gradient ascent on a representation\-theoretic energy functional\. We prove that, under random initialization, this flow drives each neuron to converge almost surely toward a single irreducible representation, while the cross\-layer Fourier coefficients achieve a rotational rank\-one alignment\. This framework provides a representation\-theoretic account of feature learning and characterizes a novel low\-rank compression phenomenon for matrix\-valued group representations\. Moreover, for Abelian groups, we provide a complete population\-level description: random initialization promotes uniform diversification across nontrivial representations and induces Haar\-uniform phases, jointly approximating the indicator via a majority\-vote mechanism\. We further prove that both phase alignment and representation competition emerge with exponential convergence rates\.

## Submission history

From: Jianliang He \[[view email](https://arxiv.org/show-email/5cd88b73/2606.02993)\] **\[v1\]**Tue, 2 Jun 2026 01:04:21 UTC \(6,164 KB\)

Similar Articles

Neural Networks Provably Learn Spectral Representations for Group Composition

Hugging Face Daily Papers

This paper provides a theoretical analysis of how neural networks learn structured representations during group composition tasks, proving that training dynamics drive neurons to converge to irreducible group representations with exponential convergence rates. The work establishes a representation-theoretic account of feature learning and characterizes a low-rank compression phenomenon for matrix-valued group representations.

Group-Equivariant Poincar\'e Convolutional Networks

arXiv cs.LG

This paper proposes Equivariant Poincaré ResNets, combining hyperbolic geometry with discrete symmetry groups to improve efficiency in learning visual representations by treating rotated features as symmetric rather than distinct hierarchical concepts.

Group Invariant Spectral Embedding

arXiv cs.LG

This paper proposes incorporating symmetries into affinity kernels for spectral embedding, proving convergence of invariant graph Laplacians on quotient manifolds with improved sample complexity.