Gill Verdon of Extropic pushes back on the idea that AI can keep getting smarter mainly by writing better code or running more of it, insisting instead that learning itself is computationally irreducible and collides with thermodynamic realities that no amount of clever software can fully bypass.
AI research itself is computationally irreducible and your rate of learning is bound by thermodynamics
yes deep learning research has a ton of headroom for RSI to discover, but our field is highly empirical!
AI certainly accelerates research progress, but takeoff seems constrained by research compute alloc
The linked report explores how automated AI research might still spark rapid progress even with modest compute growth, yet replies in the thread treat the extreme software-only acceleration case as plausible but far from certain.
Verdon's stance lines up with Extropic's work on thermodynamic chips, but the exchange offers no new measurements to quantify how soon or how hard those physical bounds would actually slow recursive improvement.
@TomDavidsonX @tszzl @MatthewJBar i also think we don't really disagree as much as you're making it sound. i think i'm just more bearish than you on priors but it's not a huge gap based on the final sentence in the post
@TomDavidsonX @tszzl @MatthewJBar i don't share the intuition that progress would be > 10x faster if only we had lots of superintelligences
so i think you're just pushing the actual argument to this number which is isomorphic to the original discussion and doesn't resolve anything
It's not just an intuition. The post argues for it. And see Ryan's comment as well.
Also see the arguments for the "one time speed up" parameter here: https://www.forethought.org/research/how-quick-and-big-would-a-software-intelligence-explosion-be
Two quick arguments: 1. If a frontier ai company had to replace it's staff with 10X fewer researchers, who were 1 SD less good at research and thought 10X slower... then progress would go way slower. So if they do the opposite, progress should speed up. 2. Suppose compute is really a bottleneck after AI R&D automation. Then hold everything fixed but add 1 OOM of extra compute to the world at that point. Now Ai progress will be 10X faster... But how do you know we're not already in the world with enough compute to go 10X faster? (Or 100X)
@TomDavidsonX @tszzl @MatthewJBar maybe you think there's a 20% chance that goes on for 10 months, i think there's a 10% chance it goes on for 3-4 months and 5% chance it goes on for 10 months
@TomDavidsonX @tszzl @MatthewJBar again want to emphasize that i don't think we disagree as much as you're making it sound
@TomDavidsonX @tszzl @MatthewJBar (2) is also wrong. you won't get 10x faster progress with 1 OOM more compute. if the world had 10x the number of people and 10x the amount of compute now, we wouldn't be making 10x faster progress
@TomDavidsonX @tszzl @MatthewJBar (1) is a flawed argument because it assumes they wouldn't be hiring up to the point where compute and human labor would be similar bottlenecks.
this obviously means you can't apply this symmetry argument because there's a reason to think current ratios are close to optimal
@TomDavidsonX @tszzl @MatthewJBar i don't share the intuition that progress would be > 10x faster if only we had lots of superintelligences
so i think you're just pushing the actual argument to this number which is isomorphic to the original discussion and doesn't resolve anything
@EgeErdil2 @tszzl @MatthewJBar What do @EgeErdil2 and @MatthewJBar think of the arguments against bottlenecks here: https://www.lesswrong.com/posts/XDF6ovePBJf6hsxGj/will-compute-bottlenecks-prevent-a-software-intelligence-1
You often point out that compute *could* be a bottleneck but I haven't seen you engage in the strongest arguments against this
@TomDavidsonX @tszzl @MatthewJBar (1) is a flawed argument because it assumes they wouldn't be hiring up to the point where compute and human labor would be similar bottlenecks.
this obviously means you can't apply this symmetry argument because there's a reason to think current ratios are close to optimal
It's not just an intuition. The post argues for it. And see Ryan's comment as well.
Also see the arguments for the "one time speed up" parameter here: https://www.forethought.org/research/how-quick-and-big-would-a-software-intelligence-explosion-be
Two quick arguments: 1. If a frontier ai company had to replace it's staff with 10X fewer researchers, who were 1 SD less good at research and thought 10X slower... then progress would go way slower. So if they do the opposite, progress should speed up. 2. Suppose compute is really a bottleneck after AI R&D automation. Then hold everything fixed but add 1 OOM of extra compute to the world at that point. Now Ai progress will be 10X faster... But how do you know we're not already in the world with enough compute to go 10X faster? (Or 100X)
@TomDavidsonX @tszzl @MatthewJBar again want to emphasize that i don't think we disagree as much as you're making it sound
@TomDavidsonX @tszzl @MatthewJBar i also think we don't really disagree as much as you're making it sound. i think i'm just more bearish than you on priors but it's not a huge gap based on the final sentence in the post
@TomDavidsonX @tszzl @MatthewJBar (the above image is from smth i wrote last year)
so if you take that literally and think current progress rates are ~ 3x/yr, i'm saying we can get ~ 3x/month for several months in a row with 10% chance
@TomDavidsonX @tszzl @MatthewJBar i also think we don't really disagree as much as you're making it sound. i think i'm just more bearish than you on priors but it's not a huge gap based on the final sentence in the post
@GillVerd How close is current AI research rate to theoretical limit?
Gill Verdon of Extropic pushes back on the idea that AI can keep getting smarter mainly by writing better code or running more of it, insisting instead that learning itself is computationally irreducible and collides with thermodynamic realities that no amount of clever software can fully bypass.
AI research itself is computationally irreducible and your rate of learning is bound by thermodynamics
yes deep learning research has a ton of headroom for RSI to discover, but our field is highly empirical!
AI certainly accelerates research progress, but takeoff seems constrained by research compute alloc