Report
Google says Streaming DiLoCo cuts training’s network chatter roughly 65-fold in MaxText tests
Google says Streaming DiLoCo keeps distributed AI training moving by streaming parameter fragments over constrained links.
TLDR
Less talking, more training: Google says Streaming DiLoCo is available in MaxText v0.2.5, streaming parameter fragments while TPUs keep computing. In its Qwen3-32B test across two TPU slices, cross-slice traffic fell roughly 65-fold versus standard data-parallel training. Google reports a 9.57× speedup with bandwidth artificially capped at 10 Gbps; without throttling, the methods ran at similar speeds.
Combined views
—
2 Sources, first seen ago
Comments