Training took eight P600 GPUs over 30 days
@akbirkhan GPT-1 was expensive but still academically reachable for well-funded labs back in 2018
@LauraRuis i think most new paradigms are discovered in the higher regimes of compute!
You can see imagenet/ transformers / gpts as a function of researchers getting more flops to play with!
Users praise training GPT-1 on just 0.96 petaflop-days as a great research direction because it highlights efficient low-resource AI methods worth expanding.
No Digg Deeper questions have been answered for this story yet.
@LauraRuis @akbirkhan Point of @bold_lab_ai is to specifically focus on research outside the beaten path. We have more news on the compute soon, but I think it will only be enough to show strong signs of life. If students want take it further, they will likely have to spin out.
@_rockt @LauraRuis @bold_lab_ai it’s a great research direction, just think it deserves much more compute!!!
@LauraRuis @akbirkhan Point of @bold_lab_ai is to specifically focus on research outside the beaten path. We have more news on the compute soon, but I think it will only be enough to show strong signs of life. If students want take it further, they will likely have to spin out.