PufferLib 4.0 multigpu is pretty good. 6.6x scaling on 8 GPUs worst-case with a 32x param policy. >100M sps peak on 8x RTX5090
4:06 PM · Jun 21, 2026 · 3.1K Views
Sentiment
Sentiment building, check back later.
Cluster Engagement
Digg Deeper
No Digg Deeper questions have been answered for this story yet.
Posts from X
Most Activity
Most Activity
VIEWS746LIKES10REPLIES1
Joseph Suarez 🐡@jsuarez
The weights and grads are in contiguous memory, so synchronization is a single nccl reduce. No additional overhead for coalescing etc. This is just a perf test, breakout already trains in 4 seconds with 1 GPU.
Joseph Suarez 🐡@jsuarez
PufferLib 4.0 multigpu is pretty good. 6.6x scaling on 8 GPUs worst-case with a 32x param policy. >100M sps peak on 8x RTX5090
3hViews 746Likes 10Bookmarks 0