Tim Dettmers#61
🚀 GLM-5.1-HighSpeed is live: 400 tokens/s — a new speed ceiling for flagship-tier LLM APIs.
Not a smaller model traded for speed. A flagship from @Zai_org that's also the fastest.
📖 Full technical deep-dive 👇 https://www.tilert.ai/blog/speed-as-the-next-scaling-law.html
Many users praised Zai Org's GLM-5.1-HighSpeed launch for its record 400 tokens-per-second speed as an impressive milestone enabling real-time uses, while a few questioned its cost-effectiveness and hype compared to faster alternatives.
TileRT rebuilds inference execution from the ground up:
- Persistent Kernel — full graph compiled AOT into one GPU-resident Engine Kernel, launched once - Tile Pipeline — compute, async IO, comms as tile-level tasks; intermediates flow through registers & shared memory, not global memory - Heterogeneous Worker — across 8-GPU NVL: GPU0 runs Sparse Indexer, GPU1–7 run MLA
@zRdianjiao @Zai_org Animated overview of GLM-5.1-HighSpeed engine With DeepManim !
@zRdianjiao @Zai_org Holy, thats fast. Might as well be instant.
@zRdianjiao @Zai_org You will either: - Getting things done really fast - Burning money blazing fast
@zRdianjiao @Zai_org When will it be available for GLM Coding Plan subscribers?
@zRdianjiao @louszbd @Zai_org I love glm models so much, I hope you guys can stay competitive for the long term.
@zRdianjiao @Zai_org Any chance to apply for getting access via coding plan pro tier?
@zRdianjiao @Zai_org Will this be available for rehosting by other orgs? Would love fast model like this hosted over here for blazing fast responses in some use cases
@zRdianjiao @Zai_org Congrats team. Alephant AI Gateway has been adapted http://Z.AI, the speed is a challenge for Alephant.
@zRdianjiao @Zai_org coding plan ? the pro one ?
pretty please :D ! With sugar on top and my gratitude :D
@zRdianjiao @Zai_org Will this model perform similarly to the standard GLM 5.1, or is it optimized more for speed with reduced performance?
400 tokens/s from a flagship-tier model is the kind of infra milestone that changes product design, not just benchmarks.
When latency drops this far, “AI features” stop feeling asynchronous and start feeling native.
Curious to see: • sustained throughput under load • reasoning quality at max speed • pricing efficiency vs GPT-5 / Gemini / Claude • real-world agent performance
The speed race just became serious.
@zRdianjiao @Zai_org Is this going to be 400 t/s at 200k context?
@zRdianjiao @Zai_org Speed like that is exciting but in my experience flagship models often trade off some depth in nuanced responses when pushed to those limits. Is there an answer for that here?
@zRdianjiao @Zai_org you all are absolutely cooking.
@zRdianjiao @Zai_org WOW
@zRdianjiao @Zai_org 400 tps on a flagship is wild. Finally enough headroom for real-time vision control on my robot arm without choking on latency. Might ditch local quantized runs for this.
@zRdianjiao @Zai_org The next AI race won’t just be about intelligence, it’ll be about speed at scale.
@zRdianjiao @Zai_org speed as a scaling law is interesting framing. inference speed compounds differently than parameter count. faster responses change the loop structure of agentic workflows more than a smarter-but-slower model does.
@zRdianjiao @Zai_org @VictorTaelin speed?