/Tech1h ago

Simo Ryu, Stable Diffusion LoRA creator, sparks debate over whether the original Transformer's encoder stack is inelegant

Story Overview

Simo Ryu shared the canonical encoder-decoder diagram from the 2017 Attention Is All You Need paper while crediting co-author Noam Shazeer, prompting replies that call the encoder stack inelegant and note its absence from most current large models.

736022.4K

#72

Original post

Simo Ryu@cloneofsimo#957inTech

Ankith 🐋/acc@dhtikna

What has shazeer even contributed at deepmind?

6:04 AM · Jun 18, 2026 · 3.7K Views

Industry Shift

Decoder-only designs now dominate

Replies point out that decoder-only architectures have become the default for frontier generative models because they train more simply on unlabeled text and generalize better in zero-shot settings.

Open Question

Encoder necessity stays open

No quantitative evidence or new benchmarks appear in the thread, so the precise drawbacks of the original dual-stack design remain a matter of ongoing architectural taste rather than settled fact.

Sentiment

Many users dismissed Simo Ryu's Transformer diagram crediting Shazeer as inelegant and replied with insults, while others noted the interesting pace of progress.

Pos

25.0%

Neg

75.0%

5 comments with sentiment.

Cluster Engagement