@srush_nlp I hate to say it but I think any architecture that processes a set of tokens through alternating per-token FFNs and inter-token communication layers I'm going to call a transformer. you don't even need attention
@Krubbled Meme of guy sitting at the “LSTMs are Transformers” table
@srush_nlp ball knower.
I circled all the Transformer parts of k3. (I'm a sore loser).
Positive users praise the expertise spotting Transformer elements in Kimi K3 while negative users dismiss the architecture distinction as insignificant and mock it via memes.
Based on 3 visible X reactions from 6 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@srush_nlp I hate to say it but I think any architecture that processes a set of tokens through alternating per-token FFNs and inter-token communication layers I'm going to call a transformer. you don't even need attention
@Krubbled Meme of guy sitting at the “LSTMs are Transformers” table
@srush_nlp ball knower.
I circled all the Transformer parts of k3. (I'm a sore loser).
Critical observation
Critical observation
I circled all the Transformer parts of k3. (I'm a sore loser).
Time for everyone’s favorite game! Is it a Transformer?
Critical observation
I circled all the Transformer parts of k3. (I'm a sore loser). https://twitter.com/srush_nlp/status/2077892487411052915
Positive users praise the expertise spotting Transformer elements in Kimi K3 while negative users dismiss the architecture distinction as insignificant and mock it via memes.
Based on 3 visible X reactions from 6 accounts; directional sample.
Ask a question below.
Published answers will appear here.
