Researcher Ege Erdil supported the claim, citing physical bottlenecks.
@tszzl I don't think it's a good development. I continue to think that RSI is an overrated risk vector due to data and compute bottlenecks, and that slowing down AI would accomplish little at enormous cost.
now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development
Many users rejected the claim that recursive self-improvement is an overrated AI risk, calling the dismissal vague, weak, and blind to large efficiency gains still possible in deep learning.
@MatthewJBar I think you’re wrong and there’s 1,000x efficiency gains leftover in deep learning research that could lead to much smarter faster more agentic models given the same inputs
@tszzl @MatthewJBar i agree RSI *can* change things dramatically. that's again an extremely weak claim that:
1. presupposes "RSI" exists 2. qualifies the "change things dramatically" prediction with "can" (which makes it impossible to disagree with, especially conditional on 1)
@tszzl @MatthewJBar matthew is just saying RSI is overrated as a *risk vector*, as in, he thinks it's not going to happen because there are other inputs that go into improving AI systems that will become bottlenecks with abundant researcher effort
your claim doesn't respond to that at all
@tszzl @MatthewJBar i agree RSI *can* change things dramatically. that's again an extremely weak claim that:
1. presupposes "RSI" exists 2. qualifies the "change things dramatically" prediction with "can" (which makes it impossible to disagree with, especially conditional on 1)
@tszzl @MatthewJBar in a world where r&d progress in AI is itself compute-bottlenecked, it can simultaneously be true that:
1. we're far from the "landauer limit" of efficiently using compute and data during training, inference, etc
2. we can't close the gap to it by scaling cognitive effort alone
what ML researchers do is improve compute efficiency and data efficiency. if you believe we’ve already covered most of the ground, then you’ve got little to fear. if you think we aren’t close to the “Landauer limit” (using this entirely metaphorically) of model training, then RSI can change things dramatically
@tszzl @MatthewJBar wouldn’t that just risk letting other countries lead?
@tszzl @MatthewJBar that's such a vague claim
@tszzl @MatthewJBar I'm interested to know how we a priori know this.
The scaling laws seem pretty dominant, don't they?
@tszzl @MatthewJBar Yah someone with an m5 and 16GB is using codex to build a scratch model comparable to codex itself in someways but completely different in the solution space it produces - this is rsi though
@MatthewJBar @tszzl I very much understand your point, but are you saying you think even within twenty years from now it's unlikely there'll be a major acceleration of capabilities growth due to RSI?
@MatthewJBar @tszzl It’s more of a cost-benefit question.
Late a decade or two on curing cancer vs. reducing large safety event by XX%.
Reasonable trade off given our civilization’s overall age.
@tszzl @MatthewJBar we're going to create super intelligence and the first thing it'll say is
"lol, here's how you build super intelligence with the power of a microwave, my dudes"
@itspublu @tszzl @MatthewJBar I guess that's why he said "mutual conditional"
@MatthewJBar @tszzl I keep thinking about what if pausers had got the 6 month pause they asked for in March 2023. There'd have been a 6-month delay for... no benefit whatsoever.
Researcher Ege Erdil supported the claim, citing physical bottlenecks.
@tszzl I don't think it's a good development. I continue to think that RSI is an overrated risk vector due to data and compute bottlenecks, and that slowing down AI would accomplish little at enormous cost.
now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development