GLM 5.3 Flash Runs Untuned on Dual DGX Stations
Alec Fong shares untuned inference results for the model on his hardware setup.
Alec Fong posted that GLM 5.3 Flash will likely serve as his team's new daily driver. He ran the model untuned on two DGX stations with tensor parallelism of 2. The setup delivered 881 tokens per second at context length 64 and 232 tokens per second at context length 1. Fong noted the model supports vision and could handle four to eight simultaneous users at longer contexts. A retweet of the post pointed to remaining opportunities in prediction heads and unoptimized kernels by comparing the machine's cost performance to recent discounts on smaller systems.
Combined views
91.7K
3 posts, first seen 15h ago
Reactions
3 postsAF 
Alec Fong@alecqfongGLM 5.3 Flash will likely be my teams new daily driver Untuned I'm getting on 2xDGX Station TP=2 881 tok/s C=64 232 tok/s C= 1 At long context realistically can have 4-8 simultaneous users. + it supports vision!
- likes: 700
- replies: 44
- bookmarks: 209
- reposts: 22
RO Roy@usr_bin_roygbivto illustrate the opportunity that still lies in prediction heads and how unoptimized kernels are this ~$120k machine performs the same as two six thousands that until a month ago were $20k ($30k now)
- likes: 57
- replies: 1
- bookmarks: 30
- reposts: 2
RO Roy@usr_bin_roygbivto illustrate the opportunity that still lies in prediction heads and how unoptimized kernels are this ~$120k machine performs the same as two six thousands that until a month ago were $20k ($30k now)
- likes: 57
- replies: 1
- bookmarks: 30
- reposts: 2