I circled all the Transformer parts of k3. (I'm a sore loser).
Time for everyone’s favorite game! Is it a Transformer?
is k3 actually a transformer? why not consult The Chart!
Positive users praise the Kimi K3 Transformer diagram and its sharers as expert, while negative users dismiss the sparse attention claims as a silly kernel method and question whether the diagram misleads compared to loss curves.
Based on 4 visible X reactions from 14 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@srush_nlp They made attention sparse and all they got was this silly kernel method.
@srush_nlp honestly the labeled diagram is doing more work than the loss curves tho
@srush_nlp ball knower.
Positive users praise the Kimi K3 Transformer diagram and its sharers as expert, while negative users dismiss the sparse attention claims as a silly kernel method and question whether the diagram misleads compared to loss curves.
Based on 4 visible X reactions from 14 accounts; directional sample.
Ask a question below.
Published answers will appear here.
