Huawei Ascend SuperPoD Trains 500B Models In 90 Days At Scale
Reactions from ranked influencers
3 posts
>50% training throughput is from Huawei's own claims then again they claimed it for 384 too, and in reality we know that with a MoE like V4 people get 30%. On the other hand, ByteDance has a paper where they get 40% on a 2.4T MoE. So I think 35ish is doable @NunoSempere
@NunoSempere is this tru or slop @zephyr_z9
@NunoSempere A more interesting question is how far their FP4 implementation goes for training