@eliebakouch Yes MSA makes more sense
elie@eliebakouch
@teortaxesTex doesn't have 1M context! model without sparse attention is ok at 256k (also you can do GQA -> MSA)
12:22 AM · Jul 6, 2026 · 305 Views
Sentiment
Sentiment building, check back later.
Cluster Engagement
Digg Deeper
No Digg Deeper questions have been answered for this story yet.
Posts from X
Most Activity
Most Activity
VIEWS239RETWEETS1REPLIES1
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex
@giffmana My point about architecture transition is that *this model* might warrant a post-training project to adapt it to some sparsity (like Minimax's MSA, which starts from a GQA checkpoint)
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex
@giffmana it does not, I disapprove of the architecture independently of benchmarks. There just was no point to this model before, because DSV4-Flash(preview) exists. Now it's frustrating because scores are good but it'll be handicapped on long context/high batch size
3hViews 239Likes 1Bookmarks 0
LIKES2
Lucas Beyer (bl16)@giffmana
@teortaxesTex ahhh sorry yeah, I read you the wrong way around.
3hViews 71Likes 2