Commentary on X
@_xjdr @spyced d4 is cheap and functional enough that I can use it day in day out w/ the api and justify the cost of doing so - that was the real threshold change for me. sure, glm5.2 is better, but costs more. ds4 price is right w/ minimum required functionality & w/o coding plan rate limits.
10:21 AM · Jul 16, 2026@_xjdr Hoo-rah Ameriboos have been chasing the "America innovates, China follows" narrative and now have to deal with the cognitive dissonance of it all. The narrative will now congeal on "well, they copied us, so of course we should copy them" and everything will be fine.
9:32 AM · Jul 16, 2026@_xjdr If the recipe was this obvious and out in the open, llama4 falling short wasn't a research gap, it was meta just not doing it.
9:33 AM · Jul 16, 2026my not sure why there is surprise, concern with the TM approach. they did with inkling did exactly what i said a western lab should do. DSV3 shaped but with at least 4:1 sliding window, train on k2* rollouts make it at least 1T params. use muon and mup and train it on gb300s. this is _literally_ nmoe. this is what meta should have done from the start with llama4 and beyond. now what i want to see (im being selfish now) is something 3T+ , more expert sparsity and kvcache compression, a base model release, frontier RL polish and a distillation paper. at higher sparsity, DSV3 shaped architecture is _fine_ for 3T+ (read not the limiting factor) for the near term .
9:09 AM · Jul 16, 2026View all
maybe this is k3 but we will have to wait and see (fingers crossed)
9:09 AM · Jul 16, 2026