DRMoET may improve load balancing in mixture-of-experts training

A post highlights Distributionally Robust Mixture-of-Experts Training (DRMoET) as a load-balancing improvement that seems to do better than DeepSeek's aux-loss-free approach without a throughput cost. It also quotes a claim that DRMoET improves mixture-of-experts performance by reducing the penalty of imperfect routing.