Aidan Clark argues AGI pretraining is now restricted to six industry labs with massive compute resources
Will Depue says private compute will dictate scientific progress
#64Nathan Lambert@NATOLAMBERTSomething that's squashed in this is that other things impact the evolution of AI than just the few labs with mega compute.
There are many social dynamics, policy, diffusion, etc and there's a substantial unaddressed opportunity for impact here. Open science/models one way.
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
not even just concentration of power, but over concentration of talent in the AI era is going to make a lot of the positive potential take much longer to roll out.
Nathan Lambert@natolambertSomething that's squashed in this is that other things impact the evolution of AI than just the few labs with mega compute. There are many social dynamics, policy, diffusion, etc and there's a substantial unaddressed opportunity for impact here. Open science/models one way.
@willdepue they're make technical progress, but much of science is communicating your ideas with a community and progressing collective knowledge, so in the near term doesn't really seem like the labs are going to participate much in that.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
@willdepue Tradition doesn't show that these labs are particularly open and actually doing science these days, culture takes a very long time to change.
will depue@willdepue@natolambert i'm confused how we'd end up in a world where labs can make large technical progress in academic fields but are limited by communicating them. that seems like the easy part, comparatively. writing the paper is easier than producing the result & verifying it?
#83rohan anil@_AROHAN_This is very true. This is why the best models are from Anthropic earlier this year as they had most compute compared to everyone else.
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup.
The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
@eliebakouch Post-training can mean many things! There are some parts for which I’d say it’s just as true. There are others which don’t have the same fundamental gap, though I think it’s worth asking whether, empirically, the non-frontier labs have led the charge here.
elie@eliebakouch@_aidan_clark_ would you say this is true as well for post training?
academics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
@natolambert i'm confused how we'd end up in a world where labs can make large technical progress in academic fields but are limited by communicating them. that seems like the easy part, comparatively. writing the paper is easier than producing the result & verifying it?
Nathan Lambert@natolambert@willdepue they're make technical progress, but much of science is communicating your ideas with a community and progressing collective knowledge, so in the near term doesn't really seem like the labs are going to participate much in that.
#254will depue@WILLDEPUEy'all don't realize how big the compute gap is about to be. all things considered, labs haven't had that much more compute (esp. divided by # of bets) so far. all those GPUs are finally landing. expect progress to be bonkers this year, if only for that. table stakes are 10B now
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
@tenobrus i think there will be considerable collaboration between humans and models for a while. as i mentioned, data collection, harnesses, ... generally what to point the models at is a hard problem, recognizing big results. the long middle
Tenobrus@tenobrushm. but will it really be "splitting" the credit? even if it is factually true that gpt 6.2 is only dong half the work, or that the ML / data / scaffolding of the scientists unlocked the research capabilities, it seems clear that "credit" for the discoveries won't meaningfully accrue to the individual humans in anything like the same way. who is getting "credit" for the erdos problem? the PR cycle is mostly accruing to "GPT" and "OpenAI" as abstract entities, not specific researchers within the org or the many mathematicians who have helped push forward ai-assisted math research. and like... that doesn't particularly seem incorrect to me. it's not the case that the proof *should* have a different name in the list of authors. it seems like as research is automated, researchers and research reputation and careers are pretty clearly decentered. i can see many researchers being very excited about that for the sake of understanding and solving problems anyway, but i doubt it will be quite the clamoring or "excellent trade" you describe ? what physicists are buying by doing this is both peering deeper into the nature of the universe *and also* ensuring they never personally win the nobel prize (or at least if they do, it's in a world where the sense of prestige has shifted dramatically, where the personal impact and expertise required for such discoveries drops through the floor)
#254will depue@WILLDEPUEit’s not so much that GOMAX will pick and choose what to make progress on — for the most part)
but they will pick and choose what evals to track, what north star proxies for 'general intelligence' they’ll focus on, and what big announcements to push for. this effects where compute & time is spent
access to the latest models will also be increasingly gated. you want GPT-6.2-SuperPro with max inference compute, you’ll need to be inside the lab. don’t forget, you get infinite tokens and access to the full research cluster if you’re there.
and, don’t forget, is research cluster utilization is abysmal. anyone who’s run a research cluster knows how hard it is to constantly have experiments running. sometimes up to 20-30% of a cluster could be doing nothing. that’s millions, maybe billions in compute, just sitting there idle. if you’re a researcher, nobody minds if you take unscheduled GPUs and put a few zettaflops into solving discrete geometry problems on the weekends!
i think that the OpenAI foundation/other philanthropic organizations will help fund open access to compute. but anyone inside the labs will maintain a permanent significant advantage to those outside: unreleased models, free tokens, model customization
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
^GOMAX ~ google openai anthropic meta anthropic spacex
we really need a good lab shorthand
will depue@willdepueit’s not so much that GOMAX will pick and choose what to make progress on — for the most part) but they will pick and choose what evals to track, what north star proxies for 'general intelligence' they’ll focus on, and what big announcements to push for. this effects where compute & time is spent access to the latest models will also be increasingly gated. you want GPT-6.2-SuperPro with max inference compute, you’ll need to be inside the lab. don’t forget, you get infinite tokens and access to the full research cluster if you’re there. and, don’t forget, is research cluster utilization is abysmal. anyone who’s run a research cluster knows how hard it is to constantly have experiments running. sometimes up to 20-30% of a cluster could be doing nothing. that’s millions, maybe billions in compute, just sitting there idle. if you’re a researcher, nobody minds if you take unscheduled GPUs and put a few zettaflops into solving discrete geometry problems on the weekends! i think that the OpenAI foundation/other philanthropic organizations will help fund open access to compute. but anyone inside the labs will maintain a permanent significant advantage to those outside: unreleased models, free tokens, model customization
#254will depue@WILLDEPUEcan we make this a thing
will depue@willdepue^GOMAX ~ google openai anthropic meta anthropic spacex we really need a good lab shorthand
@_arohan_ 🥇
rohan anil@_arohan_This is very true. This is why the best models are from Anthropic earlier this year as they had most compute compared to everyone else.
#279Luca Soldaini 🎀@SOLDNIno compute for fully open research no way chinese don’t close up more, already say ~nothing on data
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
#368Yi Ma@YIMATWEETSMostly agree.
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
#567Leo Boytsov@SRCHVRSMe, reading this post, is "in the pocket of industry" a metaphor, a simile, or is just mildly derogative?
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
#716elie@ELIEBAKOUCH@_aidan_clark_ :(
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
#716elie@ELIEBAKOUCH@_aidan_clark_ would you say this is true as well for post training?
elie@eliebakouch@_aidan_clark_ :(
#716elie@ELIEBAKOUCH@natolambert > especially finically driven
i'd guess that for a lot people it's also a "compute driven" decision
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
#867Alexander Doria@DORIALEXANDERExactly why the frontier should be distributed.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
The upside: frontier labs have until now been suspiciously advancing in directions where they had deep internal expertise at the core. As we move toward open-endedness, this might be the actual bottleneck.
Alexander Doria@DorialexanderExactly why the frontier should be distributed.
#1153Florian Brand@XEOPHONOpen source, open weight and especially open science are more important than ever.
Participating in and accelerating the open ecosystem means everyone wins 🫡
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.