Teortaxes Lists Attention Tricks in Reply to Yoavgo
Reply suggests converting GQA to MLA plus block sparse attention tricks.
Teortaxes, a pseudonymous AI developer known as DeepSeek's most vocal online booster, replied to @yoavgo with concrete suggestions. The post states that GQA can be converted to MLA and notes that MLA was not done in 2026. It adds that a block sparse attention mechanism can be post-trained and lists several other tricks as further options. The reply stands as one visible comment in the ongoing public thread about model architecture choices.
Combined views
342
1 post, first seen 14h ago
Teortaxes Lists Attention Tricks in Reply to Yoavgo
Reply suggests converting GQA to MLA plus block sparse attention tricks.
Teortaxes, a pseudonymous AI developer known as DeepSeek's most vocal online booster, replied to @yoavgo with concrete suggestions. The post states that GQA can be converted to MLA and notes that MLA was not done in 2026. It adds that a block sparse attention mechanism can be post-trained and lists several other tricks as further options. The reply stands as one visible comment in the ongoing public thread about model architecture choices.
Reactions
1 postT( 
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex@yoavgo well, for example you can convert GQA to MLA (I don't think they did MLA in 2026) you can post-train a block sparse attention lots of tricks
- likes: 2
- replies: 0
- bookmarks: 3
- reposts: 0