Core Automation details AI training run across clusters about 9,000 miles apart
Core Automation describes two sites with different GPUs and parallelism schemes connected by a bidirectional 25 Gbps link.
TLDR
Core Automation reports training a roughly 32-billion-parameter mixture-of-experts model across differently equipped GPU clusters about 9,000 miles apart. In its tests, the two-site run reached about 1% lower bits per byte than standard training at equal compute on both evaluations. Separately, the company reports that an eight-learner butterfly-mixing experiment matched standard training. Both comparisons are results reported by the company for its own tests.
Combined views
50K
9 Sources, first seen ago
