/AI12h ago

Yale's Zhuoran Yang proves neural networks solve group composition by learning spectral representations

The theory extends modular addition analysis to non-Abelian groups.

28321899.4K
Zhuoran Yang
Zhaoran Wang#851
Original postJason Lee#122
Zhuoran Yang@zhuoran_yang

Followup Paper -- "Neural Networks Provably Learn Group Representations: From Cyclic to Non-Abelian"

Our prior work (https://arxiv.org/abs/2602.16849) showed that NNs trained on modular addition learn **sinusoidal features** with precise **phase alignment**. But why sinusoidal? Why those phases?

Because sine and cosine are the **irreducible representations** of the cyclic group Z_p. The network is discovering the representation theory of the group it computes on.

This is not a coincidence. In our new paper, we show that all of these findings can be generalized to Abelian groups, and there is some surprise for non-Abelian groups.

In particular, we study how NNs learn the group composition task: given two group elements g1 and g2, predict g1*g2. The key insight is that NN learns to leverage **group fourier features** to solve this task.

arxiv: https://arxiv.org/abs/2606.02993 blog: https://y-agent.github.io/posts/group_composition_learning/ code: https://github.com/Y-Agent/nn-group-representation-learning

@JLiangHe @LedaW77625 @FengzhuoZhang @siyuc3141

7:46 AM · Jun 9, 2026 · 9.7K Views
Sentiment

Users find the paper on neural networks provably learning group Fourier features for composition tasks extremely interesting because it shows NNs can handle complex non-linear tasks effectively.

Pos
100.0%
Neg
0.0%
1 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS590
Zhuoran Yang@zhuoran_yang

For Abelian groups (e.g., Z_3 × Z_5, order 15), we prove that the three observations from modular addition hold in full generality:

(1) Single representation: each neuron learns ONE irreducible representation (generalized Fourier mode)

(2) Phase alignment: output phase = sum of input phases

(3) Diversification: neurons spread uniformly across all characters

Together, these yield a **flawed indicator** mechanism — giving perfect accuracy through group orthogonality.

12hViews 590Likes 1
BOOKMARKS1
Zhuoran Yang@zhuoran_yang

Now the leap to non-Abelian groups — where g1*g2 ≠ g2*g1.

For Abelian groups, irreps are 1D (scalars — complex exponentials). For non-Abelian groups, irreps become d×d **matrices**.

Example: the alternating group A_4 (order 12) has a 3D irrep — each group element maps to a 3×3 unitary matrix.

12hViews 519Likes 2Bookmarks 1
LIKES3
Zhuoran Yang@zhuoran_yang

Concrete test case: the Frobenius group C_7 ⋊ C_3 (order 21).

It has five irreps: three 1D (scalar) and two 3D (matrix-valued). Dimension check: 1²+1²+1²+3²+3² = 21 ✓

Each neuron's Fourier transform decomposes into these blocks. After training — block-sparse! Each neuron picks exactly one irrep, scalar or matrix.

(Each row is one neuron; columns are grouped by irreducible representation (three 1D blocks of width 1, then two 3D blocks of width 9 each.)

12hViews 159Likes 3
RETWEETS19
Zhuoran Yang@zhuoran_yang

Followup Paper -- "Neural Networks Provably Learn Group Representations: From Cyclic to Non-Abelian"

Our prior work (https://arxiv.org/abs/2602.16849) showed that NNs trained on modular addition learn **sinusoidal features** with precise **phase alignment**. But why sinusoidal? Why those phases?

Because sine and cosine are the **irreducible representations** of the cyclic group Z_p. The network is discovering the representation theory of the group it computes on.

This is not a coincidence. In our new paper, we show that all of these findings can be generalized to Abelian groups, and there is some surprise for non-Abelian groups.

In particular, we study how NNs learn the group composition task: given two group elements g1 and g2, predict g1*g2. The key insight is that NN learns to leverage **group fourier features** to solve this task.

arxiv: https://arxiv.org/abs/2606.02993 blog: https://y-agent.github.io/posts/group_composition_learning/ code: https://github.com/Y-Agent/nn-group-representation-learning

@JLiangHe @LedaW77625 @FengzhuoZhang @siyuc3141

12hViews 9.7KLikes 83Bookmarks 90
REPLIES3
Daniel@DanPaul49989437

@zhuoran_yang Extremely interesting. Correct me if I am wrong (I am quite far from that 🙃): so generally NNs're generally capable of handling non-abelian representations, but they struggle with identifying those structures on complex topologies?

6hViews 4
Zhuoran Yang@zhuoran_yang

But scalar "phase alignment" doesn't make sense for matrices. What replaces it?

Answer: **rank-one rotational alignment**.

(1) The d x d Fourier matrices collapse to **rank-one** (2) Rotational alignment: Fourier(output) = Constant* Fourier(input2) * Fourier(input1)

For scalars (d=1), matrix product = multiplying phases, so phase alignment was the 1D special case of a matrix alignment.

12hViews 288Likes 2
Daniel@DanPaul49989437

@zhuoran_yang *quite far from those topics

6hViews 3
Zhuoran Yang@zhuoran_yang

@DanPaul49989437 We do not go beyond to more complex algebraic/geometric structures, which are out of our current scope.

2hViews 2
Zhuoran Yang@zhuoran_yang

@DanPaul49989437 We show that NN (i) is capable of solving the group-composition task for non-abelian group (ii) achive this by having neurons **exactly learn** group Fourier features (iii) the features additionally satisfy the rank-one and rotational alignment conditions (not in Abelian groups)

2hViews 2