Researcher Questions GLM-5.2 SSD Streaming Speed On M5 Max
Reactions from ranked influencers
3 posts
I do not get it: - GLM-5.2 is 750B parameters, 40B active= 20Gb, - A top SSD can do *in theory* 15Gb/s. how can you do better than <1 token/s ?
GLM 5.2, Q2_K routed experts (effectively ~2.6 bits) running with SSD streaming on an M5 Max 128GB computer.
Google response seems reasonable.
I do not get it: - GLM-5.2 is 750B parameters, 40B active= 20Gb, - A top SSD can do *in theory* 15Gb/s. how can you do better than <1 token/s ? https://twitter.com/antirez/status/2071173841175363905
This kind of posts for instance, I wish I could summarize the responses I got and fix the mistake I wrote.
I do not get it: - GLM-5.2 is 750B parameters, 40B active= 20Gb, - A top SSD can do *in theory* 15Gb/s. how can you do better than <1 token/s ? https://twitter.com/antirez/status/2071173841175363905