Reactions from ranked influencers
7 posts
@eliebakouch @thinkymachines i was already excited today about the thought of kimi-k3 j-space but Kimi-K2-2.7 and Inkling are also interesting
@thinkymachines (btw @thinkymachines folks would be great to have the model on open router, initially i wanted to do something fun and ask inkling to replicate jspace on itself to test the model aha 🙏)
@eliebakouch @thinkymachines fascinating! I wonder why...
we replicated anthropic jspace analysis on @thinkymachines Inkling new 1T model! it seems to be an outlier: where other models split into near-orthogonal sensory/workspace/motor blocks, inkling keeps roughly one geometry across the whole stack (early-late CKA ~0.8 vs ~0.5 elsewhere) we also look computed the J-space of @poolsideai's laguna XS 2.1 in bf16 vs nvfp4 to test the impact of quantization. result: almost none. the quantized model has the same jlens space as the non-quantized one both jspace checkpoint are up on hugging face!
Today, we are introducing Inkling. Inkling reasons efficiently across text, image, and audio modalities. We are making the full weights available. https://thinkingmachines.ai/news/introducing-inkling/ Available today for fine-tuning on Tinker. Play with it in the Inkling Playground. 🧵
oh also there is some jspace for larger MoE models but fable made a big approximation and only computed it for very few layers 😭 i'm running those experiments on the side, mostly letting the model decide what makes sense. it downloaded the nvfp4 checkpoint of inkling just because it was faster. it was transparent about potential errors this might introduce, but the model didn't think of running the experiment on a smaller model like poolside to check the impact (i had to tell it), same for the impact of prompt count!
@johnschulman2 @thinkymachines aha i'm not an expert tbh so i actually don't have strong intuition here, waiting to see what more knowledgable ppl think about it 👀 gemma 4 models also have very weird shapes btw put all the heatmap in this vibecoded site https://eliebak.com/viz/jspace-open