PrimeIntellect also claims up to 34% faster hosted training with prime-rl in tests on two Qwen3 models.
PrimeIntellect introduced Prime Mega MoE, saying it made DeepSeek’s Mega MoE kernel trainable. In tests on two Qwen3 models, it reports up to 55% faster MoE training than TorchTitan and up to 34% faster hosted training with prime-rl.
308
11 Sources, first seen ago
#13
#13
@PrimeIntellectIntroducing Prime Mega MoE. We made DeepSeek's Mega MoE kernel trainable. The results on two Qwen3 models: • Up to 55% faster MoE training than TorchTitan • Up to 34% faster hosted training with prime-rl
@samsja19we made our training stack significantly faster by fusing computation and communication in the MoE layer, adding backward kernels to deepseek MegaMoE available for BF16, MXFP8, and FP8 on Blackwell and Hopper GPUs
@ad0rnai@PrimeIntellect can’t stop won’t stop@PrimeIntellectIntroducing Prime Mega MoE. We made DeepSeek's Mega MoE kernel trainable. The results on two Qwen3 models: • Up to 55% faster MoE training than TorchTitan • Up to 34% faster hosted training with prime-rl@samsja19we made our training stack significantly faster by fusing computation and communication in the MoE layer, adding backward kernels to deepseek MegaMoE available for BF16, MXFP8, and FP8 on Blackwell and Hopper GPUs@ad0rnai@PrimeIntellect can’t stop won’t stop@PrimeIntellectIntroducing Prime Mega MoE. We made DeepSeek's Mega MoE kernel trainable. The results on two Qwen3 models: • Up to 55% faster MoE training than TorchTitan • Up to 34% faster hosted training with prime-rl@samsja19we made our training stack significantly faster by fusing computation and communication in the MoE layer, adding backward kernels to deepseek MegaMoE available for BF16, MXFP8, and FP8 on Blackwell and Hopper GPUs@ad0rnai@PrimeIntellect can’t stop won’t stop@PrimeIntellectIntroducing Prime Mega MoE. We made DeepSeek's Mega MoE kernel trainable. The results on two Qwen3 models: • Up to 55% faster MoE training than TorchTitan • Up to 34% faster hosted training with prime-rl@samsja19we made our training stack significantly faster by fusing computation and communication in the MoE layer, adding backward kernels to deepseek MegaMoE available for BF16, MXFP8, and FP8 on Blackwell and Hopper GPUs@ad0rnai@PrimeIntellect can’t stop won’t stop
