Emad Mostaque says a single researcher can now manage several thousand GPUs for AI training with modern automation
Historically, one researcher managed 1,000 A100 GPUs.
#156David@DAVIDSHOLZ@unixpickle yah I mean can a single researcher with agents really do a 100k GPU training job?
Alex Nichol@unixpickle@DavidSHolz If they have coding agents, probably unbounded
#167Emad@EMOSTAQUE@DavidSHolz A few years ago we had 1000+ GPUs (so 128 nodes) fine on a100s by a single researcher
With h and b chips and new failover you should be able to get to thousands comfortable
On TPUs even more i would think
David@DavidSHolzhow many gpus do you think a single researcher can handle at once for a single big training job without any help? (assume it's set up with slurm or something)
#167Emad@EMOSTAQUE@Clashluke @DavidSHolz It was a good cluster, sadly missed
(well aside from the interconnect ofc, but still)
Lucas Nestler@Clashluke@EMostaque @DavidSHolz I once used ~90% of the cluster to tokenize videos, before getting kicked for hogging preemptible compute
@DavidSHolz I was able to do a few k gpus at once solo, which I found mostly stressful but very empowering
David@DavidSHolzhow many gpus do you think a single researcher can handle at once for a single big training job without any help? (assume it's set up with slurm or something)
@DavidSHolz If they have coding agents, probably unbounded
David@DavidSHolzhow many gpus do you think a single researcher can handle at once for a single big training job without any help? (assume it's set up with slurm or something)
@EMostaque @DavidSHolz I once used ~90% of the cluster to tokenize videos, before getting kicked for hogging preemptible compute
Emad@EMostaque@DavidSHolz A few years ago we had 1000+ GPUs (so 128 nodes) fine on a100s by a single researcher With h and b chips and new failover you should be able to get to thousands comfortable On TPUs even more i would think