/Tech12h ago

Yale's Zhuoran Yang releases research proving neural networks learn spectral representations to solve group composition tasks

The mathematical proof extends the phenomenon to non-Abelian groups

28321909.7K

Original post

Jason Lee#328

Zhuoran Yang@zhuoran_yang

Followup Paper -- "Neural Networks Provably Learn Group Representations: From Cyclic to Non-Abelian"

Our prior work (https://arxiv.org/abs/2602.16849) showed that NNs trained on modular addition learn **sinusoidal features** with precise **phase alignment**. But why sinusoidal? Why those phases?

Because sine and cosine are the **irreducible representations** of the cyclic group Z_p. The network is discovering the representation theory of the group it computes on.

This is not a coincidence. In our new paper, we show that all of these findings can be generalized to Abelian groups, and there is some surprise for non-Abelian groups.

In particular, we study how NNs learn the group composition task: given two group elements g1 and g2, predict g1*g2. The key insight is that NN learns to leverage **group fourier features** to solve this task.

arxiv: https://arxiv.org/abs/2606.02993 blog: https://y-agent.github.io/posts/group_composition_learning/ code: https://github.com/Y-Agent/nn-group-representation-learning

@JLiangHe @LedaW77625 @FengzhuoZhang @siyuc3141

7:46 AM · Jun 9, 2026 · 9.7K Views

/Tech12h ago

Yale's Zhuoran Yang releases research proving neural networks learn spectral representations to solve group composition tasks

The mathematical proof extends the phenomenon to non-Abelian groups

28321909.7K

Original post

Jason Lee#328

Zhuoran Yang@zhuoran_yang

Followup Paper -- "Neural Networks Provably Learn Group Representations: From Cyclic to Non-Abelian"

Our prior work (https://arxiv.org/abs/2602.16849) showed that NNs trained on modular addition learn **sinusoidal features** with precise **phase alignment**. But why sinusoidal? Why those phases?

Because sine and cosine are the **irreducible representations** of the cyclic group Z_p. The network is discovering the representation theory of the group it computes on.

This is not a coincidence. In our new paper, we show that all of these findings can be generalized to Abelian groups, and there is some surprise for non-Abelian groups.

arxiv: https://arxiv.org/abs/2606.02993 blog: https://y-agent.github.io/posts/group_composition_learning/ code: https://github.com/Y-Agent/nn-group-representation-learning

@JLiangHe @LedaW77625 @FengzhuoZhang @siyuc3141

7:46 AM · Jun 9, 2026 · 9.7K Views

Sentiment

Users find the research on neural networks provably learning group Fourier features for composition tasks extremely interesting because of its implications for how NNs handle complex tasks.

Pos

100.0%

Neg

0.0%

1 comments with sentiment.

Cluster Engagement

Posts from X

Most Activity

Zhuoran Yang@zhuoran_yang

For Abelian groups (e.g., Z_3 × Z_5, order 15), we prove that the three observations from modular addition hold in full generality:

(1) Single representation: each neuron learns ONE irreducible representation (generalized Fourier mode)

(2) Phase alignment: output phase = sum of input phases

(3) Diversification: neurons spread uniformly across all characters

Together, these yield a **flawed indicator** mechanism — giving perfect accuracy through group orthogonality.

12h5901

BOOKMARKS1

Zhuoran Yang@zhuoran_yang

Now the leap to non-Abelian groups — where g1*g2 ≠ g2*g1.

For Abelian groups, irreps are 1D (scalars — complex exponentials). For non-Abelian groups, irreps become d×d **matrices**.

Example: the alternating group A_4 (order 12) has a 3D irrep — each group element maps to a 3×3 unitary matrix.

12h51921

LIKES3

Zhuoran Yang@zhuoran_yang

Concrete test case: the Frobenius group C_7 ⋊ C_3 (order 21).

It has five irreps: three 1D (scalar) and two 3D (matrix-valued). Dimension check: 1²+1²+1²+3²+3² = 21 ✓

Each neuron's Fourier transform decomposes into these blocks. After training — block-sparse! Each neuron picks exactly one irrep, scalar or matrix.

(Each row is one neuron; columns are grouped by irreducible representation (three 1D blocks of width 1, then two 3D blocks of width 9 each.)

12h1593

RETWEETS2

Zhuoran Yang@zhuoran_yang

But scalar "phase alignment" doesn't make sense for matrices. What replaces it?

Answer: **rank-one rotational alignment**.

(1) The d x d Fourier matrices collapse to **rank-one** (2) Rotational alignment: Fourier(output) = Constant* Fourier(input2) * Fourier(input1)

For scalars (d=1), matrix product = multiplying phases, so phase alignment was the 1D special case of a matrix alignment.

12h2882

REPLIES3

Daniel@DanPaul49989437

@zhuoran_yang Extremely interesting. Correct me if I am wrong (I am quite far from that 🙃): so generally NNs're generally capable of handling non-abelian representations, but they struggle with identifying those structures on complex topologies?

6h4

Daniel@DanPaul49989437

@zhuoran_yang *quite far from those topics

6h3

Zhuoran Yang@zhuoran_yang

@DanPaul49989437 We do not go beyond to more complex algebraic/geometric structures, which are out of our current scope.

2h2

Zhuoran Yang@zhuoran_yang

@DanPaul49989437 We show that NN (i) is capable of solving the group-composition task for non-abelian group (ii) achive this by having neurons **exactly learn** group Fourier features (iii) the features additionally satisfy the rank-one and rotational alignment conditions (not in Abelian groups)

2h2