I wonder what people really think about MoEs? It's ok, the voting is anonymous, you can select the option that you really think, deep inside your heart.
1:45 PM · Jul 4, 2026 · 17.4K Views
Meta's Lucas Beyer launches anonymous MoE poll, sparking debate over parameter scaling and routing efficiency
Story Overview
Meta AI researcher Lucas Beyer dropped an anonymous X poll to surface unfiltered opinions on Mixture of Experts models, quickly igniting replies that challenge whether intelligence truly tracks only the active parameters while total parameter counts drive loss and knowledge.
3814252226.3K
Open Question
Active versus total parameters remain unsettled
Replies stress that loss curves scale with every parameter yet performance narratives often cite only the routed subset, leaving routing efficiency and any non-knowledge intelligence open to doubt.
FYI
Beyer already flagged MoE scaling limits
Months earlier the same researcher noted that MoE is not actually scaling pilled, framing the poll as a follow-up probe rather than a sudden shift in view.
Sentiment
Positive users praise Mixture of Experts models for their elegant loss-free balancing, token-level routing, and faster loss descent, while some negative users dismiss them as ugly hacks or just ensembles with better PR.
Pos
81.2%
Neg
18.8%
9 comments with sentiment.
Cluster Engagement
Digg Deeper
No Digg Deeper questions have been answered for this story yet.
Posts from X
Most Activity
Most Activity
VIEWS3.9KBOOKMARKS8LIKES21REPLIES3
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex
I still wish we had something more globally aware than these routers MoEs are frustrating. What do you mean loss and knowledge scale with total params and "intelligence" with active? Wtf is non-knowledge-based intelligence in an LLM? That's not true humanlike sparsity.
Lucas Beyer (bl16)@giffmana
I wonder what people really think about MoEs? It's ok, the voting is anonymous, you can select the option that you really think, deep inside your heart.
5hViews 3.9KLikes 21Bookmarks 8
RETWEETS1
tender@tenderizzation
all model architectures live on a spectrum, ask yourself if MoEs are a step in the right direction
Lucas Beyer (bl16)@giffmana
I wonder what people really think about MoEs? It's ok, the voting is anonymous, you can select the option that you really think, deep inside your heart.
5hViews 3.6KLikes 47Bookmarks 6
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex
that this works okay vs learned routing is indictment enough
5hViews 332Likes 4
vik@vikhyatk
@giffmana my infra guy handles it
Lucas Beyer (bl16)@giffmana
I wonder what people really think about MoEs? It's ok, the voting is anonymous, you can select the option that you really think, deep inside your heart.
5hViews 516Likes 10Bookmarks 0
Alexander Doria@Dorialexander
@giffmana They just need more merch int love.
Lucas Beyer (bl16)@giffmana
I wonder what people really think about MoEs? It's ok, the voting is anonymous, you can select the option that you really think, deep inside your heart.
5hViews 408Likes 9Bookmarks 0
サメQCU@sameQCU
@teortaxesTex if the routing in the first few layers is *by construction* almost totally determined by the token embeddings, might as well use an arbitrary router instead of a learned router *for those*. or something!
4hViews 14Likes 2
kache@yacineMTB
@giffmana I hate them so much that I think you should just train smaller models
Lucas Beyer (bl16)@giffmana
I wonder what people really think about MoEs? It's ok, the voting is anonymous, you can select the option that you really think, deep inside your heart.
4hViews 905Likes 6Bookmarks 0
Ofer Hasson@hassonofer_
@giffmana Let's be honest, if it wasn't a "Yuck", we would have seen more in vision rather then V-MoE 🙃
5hViews 67Bookmarks 1
サメQCU@sameQCU
@teortaxesTex "in the initial several layers" actually wait this makes sense algorithmically with respect to the definition of a resnet, right? so many of the parameters in a resnet are wasted on conditioning embeddings in residualspace then reconditioning for-unembeddings...
4hViews 24Likes 2
ueaj@_ueaj
@giffmana loss free balancing is so elegant it makes MoEs elegant
5hViews 54Likes 1
serdarml@cs_serdar
@giffmana Is there a "there's something missing that will make it not yuck" option?
5hViews 45Likes 1
sasuke⚡420@sasuke___420
@sameQCU @teortaxesTex if you want to get rid of this you can use incredibly large n-gram embedding tables and superword input tokens but everybody is a coward. here is a random link from online https://arxiv.org/abs/2502.01637
4hViews 9Likes 1
サメQCU@sameQCU
@tenderizzation i like moe because it goes vroom and also improves loss descent on wallclock matched comparisons
5hViews 79Likes 3
vik@vikhyatk
@tenderizzation i want to have my cake and eat it too
4hViews 202Likes 2
Joe Fioti@joefioti
@tenderizzation MoEs with shared experts are great, shared experts can be loaded while router is computed. bubbles are skill issues
4hViews 80Likes 2
Tactrexas▶️ (DaapSaak 特推🐋粉铁 2O23 – ∞)@little_boy___
@rockstarondeck4 @teortaxesTex Many of my followers have already joined our WhatsApp group.!
Get free real-time trading alerts, investment strategies, and market forecast analysis.
Join the group 👇
➡️ Send “Join” to this WhatsApp number +12025567649
WhatsApp link👉🔗http://api.whatsapp.com/send?phone=12025567649&text=join
3hViews 2
Tactrexas▶️ (DaapSaak 特推🐋粉铁 2O23 – ∞)@little_boy___
@rockstarondeck4 @teortaxesTex Many of my followers have already joined our WhatsApp group.!
Get free real-time trading alerts, investment strategies, and market forecast analysis.
Join the group 👇
➡️ Send “Join” to this WhatsApp number +12025567649
WhatsApp link👉🔗http://api.whatsapp.com/send?phone=12025567649&text=join
3hViews 1
Aakash Kumar Nain@A_K_Nain
@giffmana Conceptually elegant, but practically frustrating
5hViews 57Likes 1
Artur Chakhvadze@norpadon
@giffmana @CSProfKGD MoEs are a beautiful idea, but the way routing and balancing are implemented in real models makes me cry
5hViews 46Likes 1
Benjamin Glickenhaus@benglickenhaus
@giffmana surprised there isn't more love for MoEs i think they're beautifuk
5hViews 39Likes 1
Load more posts