Both models support a 512K context length.
Huawei has finally credibly (?) pretrained a big LLM on Ascends. "hyper-node optimized training" suggests 950s I guess. Builds on DSA ("with SWA"). They want to prove it can be done on their hardware. What is ModAttn? (pics from Reddit, some translations are off)
openPangu 2.0 fully upgraded #HDC2026 #HDC #HarmonyOS7 #HarmonyOS #Pangu #openPangu2 #Upgrade
Users called Huawei a disgusting company in China when reacting to news of its OpenPangu 2.0 LLM trained on Ascend chips.
more chinese trillion param models are coming and better inference that would make them finally usable
Huawei has finally credibly (?) pretrained a big LLM on Ascends. "hyper-node optimized training" suggests 950s I guess. Builds on DSA ("with SWA"). They want to prove it can be done on their hardware. What is ModAttn? (pics from Reddit, some translations are off)
@teortaxesTex 华为现在在中国已经变成一个恶心的公司了，你可能还不知道
@scaling01 the speed these are dropping is getting hard to track
sounds like pricing war is next