Barnett says slowing down AI offers minimal safety benefits.
@tszzl I don't think it's a good development. I continue to think that RSI is an overrated risk vector due to data and compute bottlenecks, and that slowing down AI would accomplish little at enormous cost.
now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development
Users rejected the claim that RSI is an overrated AI risk due to bottlenecks, arguing instead for major remaining efficiency gains in deep learning research.
@MatthewJBar I think you’re wrong and there’s 1,000x efficiency gains leftover in deep learning research that could lead to much smarter faster more agentic models given the same inputs
@tszzl I don't think it's a good development. I continue to think that RSI is an overrated risk vector due to data and compute bottlenecks, and that slowing down AI would accomplish little at enormous cost.
what ML researchers do is improve compute efficiency and data efficiency. if you believe we’ve already covered most of the ground, then you’ve got little to fear. if you think we aren’t close to the “Landauer limit” (using this entirely metaphorically) of model training, then RSI can change things dramatically
@tszzl @MatthewJBar that's such a vague claim
@tszzl @MatthewJBar in a world where r&d progress in AI is itself compute-bottlenecked, it can simultaneously be true that:
1. we're far from the "landauer limit" of efficiently using compute and data during training, inference, etc
2. we can't close the gap to it by scaling cognitive effort alone
@tszzl @MatthewJBar matthew is just saying RSI is overrated as a *risk vector*, as in, he thinks it's not going to happen because there are other inputs that go into improving AI systems that will become bottlenecks with abundant researcher effort
your claim doesn't respond to that at all
@tszzl @MatthewJBar i agree RSI *can* change things dramatically. that's again an extremely weak claim that:
1. presupposes "RSI" exists 2. qualifies the "change things dramatically" prediction with "can" (which makes it impossible to disagree with, especially conditional on 1)
what ML researchers do is improve compute efficiency and data efficiency. if you believe we’ve already covered most of the ground, then you’ve got little to fear. if you think we aren’t close to the “Landauer limit” (using this entirely metaphorically) of model training, then RSI can change things dramatically
@tszzl @MatthewJBar that's such a vague claim
@MatthewJBar I think you’re wrong and there’s 1,000x efficiency gains leftover in deep learning research that could lead to much smarter faster more agentic models given the same inputs
@tszzl @MatthewJBar matthew is just saying RSI is overrated as a *risk vector*, as in, he thinks it's not going to happen because there are other inputs that go into improving AI systems that will become bottlenecks with abundant researcher effort
your claim doesn't respond to that at all
@tszzl @MatthewJBar i agree RSI *can* change things dramatically. that's again an extremely weak claim that:
1. presupposes "RSI" exists 2. qualifies the "change things dramatically" prediction with "can" (which makes it impossible to disagree with, especially conditional on 1)
@EgeErdil2 @MatthewJBar you’re right im just giving you squishy intuitions but these are my intuitions
@tszzl @MatthewJBar in a world where r&d progress in AI is itself compute-bottlenecked, it can simultaneously be true that:
1. we're far from the "landauer limit" of efficiently using compute and data during training, inference, etc
2. we can't close the gap to it by scaling cognitive effort alone
@tszzl @MatthewJBar wouldn’t that just risk letting other countries lead?
@tszzl @MatthewJBar I'm interested to know how we a priori know this.
The scaling laws seem pretty dominant, don't they?
@tszzl @MatthewJBar Yah someone with an m5 and 16GB is using codex to build a scratch model comparable to codex itself in someways but completely different in the solution space it produces - this is rsi though
@tszzl @DicksonPau @MatthewJBar The question on my mind is if scale and capability are truly tightly coupled, or if this is a partial consequence of current inefficiency.
@MatthewJBar I think you’re wrong and there’s 1,000x efficiency gains leftover in deep learning research that could lead to much smarter faster more agentic models given the same inputs
@MatthewJBar @tszzl I very much understand your point, but are you saying you think even within twenty years from now it's unlikely there'll be a major acceleration of capabilities growth due to RSI?
@MatthewJBar @tszzl It’s more of a cost-benefit question.
Late a decade or two on curing cancer vs. reducing large safety event by XX%.
Reasonable trade off given our civilization’s overall age.
Barnett says slowing down AI offers minimal safety benefits.
@tszzl I don't think it's a good development. I continue to think that RSI is an overrated risk vector due to data and compute bottlenecks, and that slowing down AI would accomplish little at enormous cost.
now on the eve of RSI it seems everyone is more mutual conditional pause agreement pilled than they used to be and that seems like a good development