Aidan Clark argues AGI pretraining is now restricted to six industry labs with massive compute resources
Will Depue says private compute will dictate scientific progress
@niloofar_mire sigh... rage bait gets the clicks, alas
Niloofar@niloofar_mire@suchenzang LOL i just saw ur tweet after i tweeted this:
#64Nathan Lambert@NATOLAMBERTSomething that's squashed in this is that other things impact the evolution of AI than just the few labs with mega compute.
There are many social dynamics, policy, diffusion, etc and there's a substantial unaddressed opportunity for impact here. Open science/models one way.
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
not even just concentration of power, but over concentration of talent in the AI era is going to make a lot of the positive potential take much longer to roll out.
Nathan Lambert@natolambertSomething that's squashed in this is that other things impact the evolution of AI than just the few labs with mega compute. There are many social dynamics, policy, diffusion, etc and there's a substantial unaddressed opportunity for impact here. Open science/models one way.
@niloofar_mire I’m working hard to try and have a little bit of light try to break through the clouds
Niloofar@niloofar_mireThere is this pervasive culture in the bay that “i am right and i am god and if u seek a different path than what i deem correct u are doomed” and that there is this one and only true way to “ai absolution”, which is extremely myopic, lopsided and limiting.
@willdepue they're make technical progress, but much of science is communicating your ideas with a community and progressing collective knowledge, so in the near term doesn't really seem like the labs are going to participate much in that.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
@willdepue Tradition doesn't show that these labs are particularly open and actually doing science these days, culture takes a very long time to change.
will depue@willdepue@natolambert i'm confused how we'd end up in a world where labs can make large technical progress in academic fields but are limited by communicating them. that seems like the easy part, comparatively. writing the paper is easier than producing the result & verifying it?
#83rohan anil@_AROHAN_This is very true. This is why the best models are from Anthropic earlier this year as they had most compute compared to everyone else.
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
@bneyshabur @keerthanpg 💪
Behnam Neyshabur@bneyshaburIt doesn’t need to be this way. It should not be this way. It should not be OpenAI/Anthropic deciding which fields get accelerated. To truly accelerate science and technology, self-accelerating AI R&D must be democratized. In fact, this is the exact motivation behind Mirendil.
#87Edward Grefenstette@EGREFENThere will be 3 kinds of scientists in the coming years: 1. The Blenderists, who cover their eyes to ignore the impact of AI. 2. AI scalers like OP(?), who think everything can be solved by making GPUs go brrr. 3. Actual researchers who embrace the tech and explore new frontiers.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
I meant to write "Benderists"!
Edward Grefenstette@egrefenThere will be 3 kinds of scientists in the coming years: 1. The Blenderists, who cover their eyes to ignore the impact of AI. 2. AI scalers like OP(?), who think everything can be solved by making GPUs go brrr. 3. Actual researchers who embrace the tech and explore new frontiers.
#107Behnam Neyshabur@BNEYSHABURIt doesn’t need to be this way. It should not be this way. It should not be OpenAI/Anthropic deciding which fields get accelerated. To truly accelerate science and technology, self-accelerating AI R&D must be democratized. In fact, this is the exact motivation behind Mirendil.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
A decade ago, @elonmusk thought DeepMind winning the AI race was close to inevitable. It is not over. There are many more twists in this story, and instead of watching the future unfold before our eyes, we should build the future we want.
Behnam Neyshabur@bneyshaburIt doesn’t need to be this way. It should not be this way. It should not be OpenAI/Anthropic deciding which fields get accelerated. To truly accelerate science and technology, self-accelerating AI R&D must be democratized. In fact, this is the exact motivation behind Mirendil.
@egrefen Especially 3 in combination with hard experimental sciences
Edward Grefenstette@egrefenThere will be 3 kinds of scientists in the coming years: 1. The Blenderists, who cover their eyes to ignore the impact of AI. 2. AI scalers like OP(?), who think everything can be solved by making GPUs go brrr. 3. Actual researchers who embrace the tech and explore new frontiers.
If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup.
The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
@eliebakouch Post-training can mean many things! There are some parts for which I’d say it’s just as true. There are others which don’t have the same fundamental gap, though I think it’s worth asking whether, empirically, the non-frontier labs have led the charge here.
elie@eliebakouch@_aidan_clark_ would you say this is true as well for post training?
#242Nathan Benaich@NATHANBENAICHregardless if this is true or not - it’s not gonna win a lotta friends 😂 there are better ways to inspire ppl :)
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
academics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
@natolambert i'm confused how we'd end up in a world where labs can make large technical progress in academic fields but are limited by communicating them. that seems like the easy part, comparatively. writing the paper is easier than producing the result & verifying it?
Nathan Lambert@natolambert@willdepue they're make technical progress, but much of science is communicating your ideas with a community and progressing collective knowledge, so in the near term doesn't really seem like the labs are going to participate much in that.
#254will depue@WILLDEPUEthis tweet is slightly breaking containment, and please, take a second to consider the seriousness in which i wrote it if it mentions 'the Eye of Stargate', but i completely stand by intended point. please pardon my mix of shitposting and earnestposting for now, article soon.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
Please pardon my shitposting on the original tweet, but I think we're agreeing, in the short term. For one, in the limit, if you believe in artificial superintelligence as a possibility, all scientific output is 'solved by GPUs' + tool access. My argument is simply that an increasing portion of scientific output, in the intermediate, is shared by researchers + lots of GPUs, as I mentioned in the thread. However, GPUs are extremely expensive and access to frontier models is kept private, which leads to what I mentioned.
Maybe this is illustrated best by the test-time compute chart shared by Noam: A direct connection from likelihood of discovery to amount of compute used. As the compute gap grows massively, the potential rate of discovery inside vs. outside the labs will grow in tandem.
I'll share a longer article on this later today.
Edward Grefenstette@egrefenThere will be 3 kinds of scientists in the coming years: 1. The Blenderists, who cover their eyes to ignore the impact of AI. 2. AI scalers like OP(?), who think everything can be solved by making GPUs go brrr. 3. Actual researchers who embrace the tech and explore new frontiers.
#254will depue@WILLDEPUEy'all don't realize how big the compute gap is about to be. all things considered, labs haven't had that much more compute (esp. divided by # of bets) so far. all those GPUs are finally landing. expect progress to be bonkers this year, if only for that. table stakes are 10B now
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
@nathanbenaich i enjoy being inflammatory
Nathan Benaich@nathanbenaichregardless if this is true or not - it’s not gonna win a lotta friends 😂 there are better ways to inspire ppl :)
@tenobrus i think there will be considerable collaboration between humans and models for a while. as i mentioned, data collection, harnesses, ... generally what to point the models at is a hard problem, recognizing big results. the long middle
Tenobrus@tenobrushm. but will it really be "splitting" the credit? even if it is factually true that gpt 6.2 is only dong half the work, or that the ML / data / scaffolding of the scientists unlocked the research capabilities, it seems clear that "credit" for the discoveries won't meaningfully accrue to the individual humans in anything like the same way. who is getting "credit" for the erdos problem? the PR cycle is mostly accruing to "GPT" and "OpenAI" as abstract entities, not specific researchers within the org or the many mathematicians who have helped push forward ai-assisted math research. and like... that doesn't particularly seem incorrect to me. it's not the case that the proof *should* have a different name in the list of authors. it seems like as research is automated, researchers and research reputation and careers are pretty clearly decentered. i can see many researchers being very excited about that for the sake of understanding and solving problems anyway, but i doubt it will be quite the clamoring or "excellent trade" you describe ? what physicists are buying by doing this is both peering deeper into the nature of the universe *and also* ensuring they never personally win the nobel prize (or at least if they do, it's in a world where the sense of prestige has shifted dramatically, where the personal impact and expertise required for such discoveries drops through the floor)
#254will depue@WILLDEPUEit’s not so much that GOMAX will pick and choose what to make progress on — for the most part)
but they will pick and choose what evals to track, what north star proxies for 'general intelligence' they’ll focus on, and what big announcements to push for. this effects where compute & time is spent
access to the latest models will also be increasingly gated. you want GPT-6.2-SuperPro with max inference compute, you’ll need to be inside the lab. don’t forget, you get infinite tokens and access to the full research cluster if you’re there.
and, don’t forget, is research cluster utilization is abysmal. anyone who’s run a research cluster knows how hard it is to constantly have experiments running. sometimes up to 20-30% of a cluster could be doing nothing. that’s millions, maybe billions in compute, just sitting there idle. if you’re a researcher, nobody minds if you take unscheduled GPUs and put a few zettaflops into solving discrete geometry problems on the weekends!
i think that the OpenAI foundation/other philanthropic organizations will help fund open access to compute. but anyone inside the labs will maintain a permanent significant advantage to those outside: unreleased models, free tokens, model customization
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
^GOMAX ~ google openai anthropic meta anthropic spacex
we really need a good lab shorthand
will depue@willdepueit’s not so much that GOMAX will pick and choose what to make progress on — for the most part) but they will pick and choose what evals to track, what north star proxies for 'general intelligence' they’ll focus on, and what big announcements to push for. this effects where compute & time is spent access to the latest models will also be increasingly gated. you want GPT-6.2-SuperPro with max inference compute, you’ll need to be inside the lab. don’t forget, you get infinite tokens and access to the full research cluster if you’re there. and, don’t forget, is research cluster utilization is abysmal. anyone who’s run a research cluster knows how hard it is to constantly have experiments running. sometimes up to 20-30% of a cluster could be doing nothing. that’s millions, maybe billions in compute, just sitting there idle. if you’re a researcher, nobody minds if you take unscheduled GPUs and put a few zettaflops into solving discrete geometry problems on the weekends! i think that the OpenAI foundation/other philanthropic organizations will help fund open access to compute. but anyone inside the labs will maintain a permanent significant advantage to those outside: unreleased models, free tokens, model customization
#254will depue@WILLDEPUEcan we make this a thing
will depue@willdepue^GOMAX ~ google openai anthropic meta anthropic spacex we really need a good lab shorthand
@willdepue Why is OpenAI deciding where it’s used i thought this was capitalism shouldn’t the market decide
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
@_arohan_ 🥇
rohan anil@_arohan_This is very true. This is why the best models are from Anthropic earlier this year as they had most compute compared to everyone else.
#279Luca Soldaini 🎀@SOLDNIno compute for fully open research no way chinese don’t close up more, already say ~nothing on data
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
#303Niloofar@NILOOFAR_MIRETbh i’m kinda sick of this academic doomerism vibe consuming all of bay area and the self-aggrandizing pov that frontier labs have. Sure a lot of exciting stuff is happening but we wouldn’t be where we are wo academia & there is sth to be said about the pursuit of curiosity.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
There is this pervasive culture in the bay that “i am right and i am god and if u seek a different path than what i deem correct u are doomed” and that there is this one and only true way to “ai absolution”, which is extremely myopic, lopsided and limiting.
Niloofar@niloofar_mireTbh i’m kinda sick of this academic doomerism vibe consuming all of bay area and the self-aggrandizing pov that frontier labs have. Sure a lot of exciting stuff is happening but we wouldn’t be where we are wo academia & there is sth to be said about the pursuit of curiosity.
@natolambert Super down to chat about it cause for long i used to think it is what it is, whatever but im actually starting to find this culture and mentality detrimental to long term growth, esp of students/younger folks.
Nathan Lambert@natolambert@niloofar_mire I’m working hard to try and have a little bit of light try to break through the clouds
#303Niloofar@NILOOFAR_MIRE@suchenzang LOL i just saw ur tweet after i tweeted this:
Niloofar@niloofar_mireThere is this pervasive culture in the bay that “i am right and i am god and if u seek a different path than what i deem correct u are doomed” and that there is this one and only true way to “ai absolution”, which is extremely myopic, lopsided and limiting.
#368Yi Ma@YIMATWEETSMostly agree.
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
@niloofar_mire Don’t argue with trolls. No way to convince them.
Niloofar@niloofar_mireTbh i’m kinda sick of this academic doomerism vibe consuming all of bay area and the self-aggrandizing pov that frontier labs have. Sure a lot of exciting stuff is happening but we wouldn’t be where we are wo academia & there is sth to be said about the pursuit of curiosity.
This assumes scientists are very bad at negotiations. Which is not an unfair assumption. But still. Can the Eye of Stargate replicate the Large Hadron Collider inside its activations? A Mach 30 grade wind tunnel? Or a sensor satellite? Or even a modest wet lab? No? oh, TOO BAD.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
#567Leo Boytsov@SRCHVRSMe, reading this post, is "in the pocket of industry" a metaphor, a simile, or is just mildly derogative?
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
@_aidan_clark_ It’s so obnoxious when big tech employees say their companies are the only place to have an impact from. Why is this such a trope? Industry shills flaunting their power fantasies and structural ignorance, all while not forgetting to try to sell you something.
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
#716elie@ELIEBAKOUCH@_aidan_clark_ :(
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
#716elie@ELIEBAKOUCH@_aidan_clark_ would you say this is true as well for post training?
elie@eliebakouch@_aidan_clark_ :(
#716elie@ELIEBAKOUCH@natolambert > especially finically driven
i'd guess that for a lot people it's also a "compute driven" decision
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.
#716elie@ELIEBAKOUCH@niloofar_mire people be like :(
Niloofar@niloofar_mireTbh i’m kinda sick of this academic doomerism vibe consuming all of bay area and the self-aggrandizing pov that frontier labs have. Sure a lot of exciting stuff is happening but we wouldn’t be where we are wo academia & there is sth to be said about the pursuit of curiosity.
@_aidan_clark_ I feel like **most** relevant PT problems can be adequately scaled down such that legitimate progress can be done on them with, say, university level compute. Personally, I think the real quantity being hoarded is what those relevant problems are.
Aidan Clark@_aidan_clark_If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup. The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.
#867Alexander Doria@DORIALEXANDERExactly why the frontier should be distributed.
will depue@willdepueacademics are unprepared for the coming world where much scientific progress is majorly a function of inference compute. whether OpenAI points the Eye of Stargate at your particular field will decide its acceleration. talent will leach away into the labs. it's already begun
The upside: frontier labs have until now been suspiciously advancing in directions where they had deep internal expertise at the core. As we move toward open-endedness, this might be the actual bottleneck.
Alexander Doria@DorialexanderExactly why the frontier should be distributed.
#1153Florian Brand@XEOPHONOpen source, open weight and especially open science are more important than ever.
Participating in and accelerating the open ecosystem means everyone wins 🫡
Nathan Lambert@natolambertFor a long time, academic researchers being at the cutting edge of new technologies has been a great social equilibrium. Neutral, unbiased technologists have been the people to spread new ideas to the world. As AI research takes off in velocity, it is also going behind closed doors. The tech industry has sewed distrust, and now they are the ones trying to tell the world about incredible changes coming. It's a big loss to a form of social contract in America. There's been a history of scientists helping society understand new technologies. There is a public service in the culture of science that I want to see continue. It's being exacerbated by feelings of FOMO, especially finically driven, where I'm seeing many people who previously wanted to be professors -- and likely still do deep down -- feel a need to conform and chase money, in a pocket of industry. I get it, I grapple with this. For those with a safety net, there will be great returns to some who choose to zag, and try to build something good, for people who need something different. For me, this is building interesting, fully-open models, to show what you can do with a variety of open weight sizes. Yes, AI's immediate future is dictated by the frontier, but it's long-term trajectory still deeply includes academic institutions and open science. Knowledge will always diffuse, but to whom? As of today, I think China is positioned to be the global home of AI research in a few years. The home of research is where ideas are accessible, spread rapdily, and are nurtured. The U.S. seems to be unwinding many institutions and relationships. The largest returns go to people who build something differentiated, at least in reputation, and a lot of people are not being shown that this path exists.