@rasbt For the size and time its not a good model. Don't understand why people are praising it Probably because of Mira...
@rasbt Looks promising. Highly encouraging as they made such large model weights open
@rasbt wait so theyre already saying its not the best model honest release at least
@rasbt Finally, a good open-weights model from a western lab
Positive users praise the open-weights release of Thinky's large Inkling MoE model as refreshing and encouraging from a western lab, while negative users dismiss it as underwhelming for its size and timing.
Based on 6 visible X reactions from 34 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@rasbt For the size and time its not a good model. Don't understand why people are praising it Probably because of Mira...
@rasbt Looks promising. Highly encouraging as they made such large model weights open
@rasbt wait so theyre already saying its not the best model honest release at least
@rasbt Finally, a good open-weights model from a western lab
@teortaxesTex Inkling we hardly knew thee 😭
@midsusnight yes, refreshing!
(it got outclassed in a few hours)
It supports a 1-million-token context window and processed 45T tokens.
@teortaxesTex Inkling we hardly knew thee 😭
@midsusnight yes, refreshing!
(it got outclassed in a few hours)
Interesting surprise drop from Thinky! The Inkling model looks pretty solid on benchmarks, and it has some little surprises in its architecture: - Small conv layers in several places - An RMSNorm for the embeddings (before the block RMSNorm) - Rel. position bias instead of RoPE
first open weight thinking machine model!! 975B total, 41B active trained on 45T tokens, 1M context, multimodal in sliding window with a 5:1 ratio and 512 size, deepseek aux-free load balancing and 2 shared experts (usually people only use 1), actually curious why the model is less sparse than kimi (~4.2% vs 3.2%). they use a short convolution after k and v, output and ffn (see plot), muon (they cite manifold muon but mention weight decay so not sure), muP, and have a very nice RL scaling curve and chain of thought! one very cool part of the release imo is how well their small variant (276B total, 12B active) performs compared to the big one. they mention they changed the pre-training data mix and recipe, very curious about those changes and to see them scaled up to ~1T (or more?) soon 👀 > "It is not the most performant model available today, closed or open. We trained Inkling for solid capabilities across the board rather than state-of-the-art performance in a single area, to serve as a foundation for the models we will train in the future." also this is really refreshing to see in a model release, huge congrats :)
(it got outclassed in a few hours)
Western open source frontier is back. This is on par with the largest of known Chinese pretrains so far (though might be outclassed in a few hours). And yes, it's another DSMoE +trained on Kimi data. But it has vision and *audio*. Unexpected good news. https://twitter.com/thinkymachines/status/2077454609551921208
Positive users praise the open-weights release of Thinky's large Inkling MoE model as refreshing and encouraging from a western lab, while negative users dismiss it as underwhelming for its size and timing.
Based on 6 visible X reactions from 34 accounts; directional sample.
Ask a question below.
Published answers will appear here.
