Technologist Tests LLM Training By Faking Comms On Single GPU Rank
Source Posts on X
Take any big LLM training setup and look at a single rank. You must run the exact same local compute kernels, but now imagine you only have one GPU and have to fake all data that would come from comms. Can you do this and still get non-NaN loss after ten steps.
Opus 4.8 fails this eval.
The Conversation on X
2 postsTake any big LLM training setup and look at a single rank. You must run the exact same local compute kernels, but now imagine you only have one GPU and have to fake all data that would come from comms. Can you do this and still get non-NaN loss after ten steps.
Opus 4.8 fails this eval.