/Tech9h ago

Yoshua Bengio proposes a global AI development moratorium if systems approach recursive self-improvement

Daniel Jeffries argues physical feedback loops naturally limit recursive takeoff

111158911.8K

#970

Original post

Daniel Jeffries@Dan_Jeffries1#970inTech

Come on. Even with your recent safety stance you know this is nonsense.

Reality is bound by time, multiplicity, compute and many, many other factors.

There is no free lunch even for neural networks.

People are recursively self improving and limited by countless factors, not the least of which is the subject object problem.

There is no faster way to know if a drug will have side effects in twenty years or whether something got demonstrably better at writing, versus something like code which has a fast feedback loop and right or wrong answers.

Fast take off is utter and complete nonsense and you know it.

At least argue from a grounding in the reality you most definitely have instead of an argument that you absolutely know is not true. It totally undermines the veracity of your message.

Yoshua Bengio@Yoshua_Bengio

If leading AI companies are indeed approaching the point of recursive self-improvement, a coordinated, verifiable, and universally applied pause is probably the only responsible solution to mitigate several major AI risks; at least until safety guarantees are developed and demonstrated. Ensuring that such a moratorium is respected would require sincere collaboration between various countries and companies, but I definitely believe it is achievable if others follow in @AnthropicAI's footsteps.

9:45 AM · Jun 6, 2026 · 1.1K Views

/Tech9h ago

Yoshua Bengio proposes a global AI development moratorium if systems approach recursive self-improvement

Daniel Jeffries argues physical feedback loops naturally limit recursive takeoff

111158911.8K

#970

Original post

Daniel Jeffries@Dan_Jeffries1#970inTech

Come on. Even with your recent safety stance you know this is nonsense.

Reality is bound by time, multiplicity, compute and many, many other factors.

There is no free lunch even for neural networks.

People are recursively self improving and limited by countless factors, not the least of which is the subject object problem.

Fast take off is utter and complete nonsense and you know it.

At least argue from a grounding in the reality you most definitely have instead of an argument that you absolutely know is not true. It totally undermines the veracity of your message.

Yoshua Bengio@Yoshua_Bengio

9:45 AM · Jun 6, 2026 · 1.1K Views

Sentiment

Many users dismiss Yoshua Bengio's call for a global pause on recursive AI self-improvement, claiming the race is already uncontrollable and the idea is pointless or deceptive.

Pos

0.0%

Neg

100.0%

5 comments with sentiment.

Cluster Engagement

Posts from X

Most Activity

VIEWS3.8KBOOKMARKS1LIKES28REPLIES5

(((ل()(ل() 'yoav))))👾@yoavgo

i think it is OK to allow recursive self improvement, but we need to be careful and make sure it is not tail recursion

Yoshua Bengio@Yoshua_Bengio

6h3.8K281

Gene Sobolev@genesobolev

@MancerAI_ @Dan_Jeffries1 @Yoshua_Bengio I only used LLMs because they're obvious starting points for labs. LLMs can do any research, but not new. To find novel ideas requires a mechanism for coming up with new hypotheses from recombining existing data. That doesn't exist. Otherwise, search is too expensive.

7h15

Gene Sobolev@genesobolev

@Dan_Jeffries1 @Yoshua_Bengio RSI, as I understand it, is a hypothesis search process wrapped in an agentic loop. Which is an optimization process over existing architectures. I don't think that optimization can generate enough novelty for takeoff, because you probably need to move away from LLMs for that.

8h137

MancerAI@MancerAI_

@genesobolev @Dan_Jeffries1 @Yoshua_Bengio And you don't think current day LLMs can do other ML research than just LLM theory?

7h23

sudoaptupdate@sudoaptupdater

@AISafetyMemes they never listened hehe

5h471

🇸🇴 🕳️ Stochastic Dreams@GenHeres123

@AISafetyMemes Einstein tried to stop Roosevelt but failed The AI race has escaped everyone's control No one can stop it Not even the godfathers of AI themselves

5h63

▄︻デ꩜ ꩜ ꩜ ═══━一@spirals432k

@AISafetyMemes Not going to happen

🌀

4h50

DLM@DLM93987699

@AISafetyMemes Slow down? Aside from a few chatbots that are completely inadequate, AI has done zero for the benefit of humans. That's why no one cares. AI is completely invisible. Slowing down? It's at a standstill right now for 99% of all humans.

4h15

Gene Sobolev@genesobolev

@MancerAI_ @Dan_Jeffries1 @Yoshua_Bengio Maybe LLMs with the right RL are the right approach. Also, I might not know enough about how the labs set up the learning environment for an agent to operate in. They have many smart people and I might be thinking about this naively.

7h10

jerk→Cell→Man→AGI→2026 Black Hole→?@saifer10278380

@AISafetyMemes @AISafetyMemes LIAR This is a lie. They are currently using AGI in recursive self-improvement.

3h7