Reactions from ranked influencers
8 posts
NVIDIA Blackwell Ultra just set a world record for pre-training DeepSeek-V3 671B, achieving 1,648 TFLOPs per GPU. This is about 3x the earlier delivered performance of the previous generation, and is a result of our extreme co-design and continuous software optimization across popular frameworks including Megatron-Core, TorchTitan, and JAX.
Can’t stap/won’t stap
NVIDIA Blackwell Ultra just set a world record for pre-training DeepSeek-V3 671B, achieving 1,648 TFLOPs per GPU. This is about 3x the earlier delivered performance of the previous generation, and is a result of our extreme co-design and continuous software optimization across popular frameworks including Megatron-Core, TorchTitan, and JAX.
Just did the mafs and it roughly checks out btw 0,385*2048*3600*24*55 = 3.74e24 (PFLOPs*H800s*training run duration) 37e9*14.8e12*6 = 3.28e24 I guess the discrepancy is due to the bf16 proportion 50K hoppers bros in shambles
Really cool how V3 is still the standard benchmark
3x
NVIDIA Blackwell Ultra just set a world record for pre-training DeepSeek-V3 671B, achieving 1,648 TFLOPs per GPU. This is about 3x the earlier delivered performance of the previous generation, and is a result of our extreme co-design and continuous software optimization across popular frameworks including Megatron-Core, TorchTitan, and JAX.