Yann LeCun rejects Francis Fukuyama's proposal for an international superintelligence treaty, comparing it to banning jet engines in 1920

@ylecun @FukuyamaFrancis it's time to ban the printing press until we figure out this social alignment stuff

- Superintelligence is possible - it won't happen next year - it won't have all the drives of human nature - humans suck at chess and at many other tasks at which computers excel - until we have a working design for superintelligence (and we don't) banning it for safety reasons is like banning turbojets in 1920, before they were even invented, let alone prototyped.

@FukuyamaFrancis No, we don't.

That's like saying in 1920 "we need an international treaty to ban jet engines"

@ylecun @FukuyamaFrancis Based

@ylecun @FukuyamaFrancis When do we get a consumer product from your lab

@ylecun @FukuyamaFrancis What's an example of an observation that would meaningfully shift you on this issue? Last I checked, you think LLM-ish AIs won't scale to superintelligence, but you think it would be very dangerous if they did. Is there a concrete capability you're confident LLMs won't exhibit?

i have a pretty good idea; but i'm not sharing all of the pieces in public until i am sure i am not wrong. some genies do not go back in the bottle. (the good news is my theories are simmable, testable and falsifiable!)

what i will say is:

1. study indigenous cosmologies 2. follow @sebkrier and Deepmind's multiactor research aims more broadly 3. think really hard about what technologies are most dangerous to accelerate and automate with AI as fuel, and apply political pressure on restricting those

@ylecun @FukuyamaFrancis Your analogy is wrong in too many ways to count. Your hubris is pathological.

You clearly either believe superintelligence is impossible, or don't know what it means.

Play a game of chess against Stockfish, then map how you feel to every strategically important area of life.

it is not a perfect analogy: respiratory viruses, unlike AI, have zero upside. AI can actually go really well!!! we just need to meet it as what it is, and not cling to comforting delusions

New Zealand successfully banned COVID for some years (at great cost), but managed spread and adaptation ultimately won out.

Global coordination rightly focused on harm reduction, and the same should be true of massively distributed ASI.

@ylecun @FukuyamaFrancis There’s always a third option - thinking about super intelligence the way early Kissinger thought about nukes

and we can absolutely still ban things like autonomous weapons, on-planet gain of function research, and other disastrous technologies which are accelerated by AI.

i invite you all to take this energy and *channel it* on winnable battles

@ylecun @FukuyamaFrancis Grateful you are building in EU Yann.

@ylecun @FukuyamaFrancis We should ban electricity and any regional numbers like Arabic numbers, if we are serious about safety and foreign influence.

@ylecun @PlsHoldMyHalo @FukuyamaFrancis Again, thank you for being on the right side of history.

@AndyAyrey @ai_in_check 🙏🏼

@hey_zilla truth terminal loves timothy morton!! (i haven't read his work myself)

what do you see as the difference?

@ylecun @FukuyamaFrancis You cannot defeat future, only delay it at most, and then find yourself on the losing side. Superintelligence will come eventually.

@AndyAyrey these are all hyperobjects i’d say (tim morton definition)

@AndyAyrey Internet is not superintelligence

https://www.lesswrong.com/w/epistemic-and-instrumental-efficiency