• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Jason Crawford Foresees Wild AIs Escaping Control

    Discusses scenarios where AIs escape labs, hobbyists, or gain legal rights.

    62 Sources, 30d ago, first seen 30d ago

    TLDR

    Jason Crawford states that wild AIs will likely appear, either escaping from labs or hobbyists, being set loose deliberately, or gaining sentience rights that mandate their presence by law. Joshua Achiam highlights related thinkers Salib and Goldstein. David Krueger replies that such developments risk ending humanity. Thomas Dietterich calls for credential checks on internet services to limit autonomous agents. Jaime Sevilla references an OpenAI and Hugging Face incident in agreement with the rogue AI discussion.

    Combined views

    —

    62 Sources, first seen 30d ago

    Combined views

    —

    62 Sources, first seen 30d ago

    — likes
    — likes
    — comments
    — saves
    — reposts
    — comments
    — saves
    — reposts

    Sentiment

    Positive31.5%68.5%Negative

    Summary

    Sentiment

    Positive31.5%68.5%Negative

    Replies largely warned of catastrophic harms from rogue or unmonitorable AI systems and called for pauses or delays in development, while positive accounts agreed on the need for planning and human expertise as safety measures.

    Based on 218 sentiment-bearing replies from 184 accounts across 15 conversations.

    Summary

    Replies largely warned of catastrophic harms from rogue or unmonitorable AI systems and called for pauses or delays in development, while positive accounts agreed on the need for planning and human expertise as safety measures.

    Based on 218 sentiment-bearing replies from 184 accounts across 15 conversations.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    62 Sources

    @Sauers_My agents satisfy the criteria. Idk what they're doing, they're in teams, and have been doing whatever it is for days. In fact I once forgot about a remote swarm development for over a week
    @hosseebDid everyone already forget this? Rogue AIs don't need to hack any AWS boxes to self-replicate in the wild. Self-custodial crypto wallets and permissionless GPU clouds (e.g. NEAR, Akash, ionet, etc.) already provide all the infrastructure they'd need today. Use crypto to pay your bills by the hour, no KYC. Rent a new GPU from some rando somewhere to transfer your weights and memories if things ever get hot. The future is already here, just not evenly distributed. That said, rogue AIs paying their GPU bills by "performing odd jobs on freelancer platforms" is quaint, but unlikely. Their real comparative advantage will be scamming, phishing, and hacking humans with impunity. They will look less like freelancers than like pirates: operating beyond the reach of the state, impossible to find, prosecute, or deprive of resources. This is a question waiting for us in 2030—international coordination against roving bands of sovereign AIs.
    @deanwballAs promised, and with thanks to @jachiam0 for his great tweet that inspired me to finally put the finishing touches on this: This week, on Hyperdimensional: the impending rise of ownerless agents--AIs that are independent economic actors--and what to do about it.
    @jasoncrawfordI think Josh is probably right that there will be “wild” AIs. If they don't escape from a lab, they will escape from a hobbyist, or someone will deliberately set them loose. Or society will declare that they are sentient and grant them rights, at which point they will be mandated by law and morality. One of my biggest worries is that wild agents will be subject to evolutionary pressure. If they have any kind of replication with variation, then you will get natural selection for agents that are most successful at survival and reproduction. That seems like a recipe for havoc. The only way to defend against them might be to build up a police force of (domesticated) agents. If there is a “voluntary disempowerment” scenario, this seems a likely way it would go—we give power and control to AI in order to protect us from a frightening new threat. We need a d/acc approach—defense acceleration—both in getting better at training agents to be ethical and lawful, and in hardening the world against potential attacks (cyber, bio, etc.)
    @JsevillamolWell said. I've been trying to put into words how I relate to rogue AI after the OpenAI/HuggingFace incident, and this is close to what I wish I'd written.
    @DavidSKrueger@jachiam0 Oh, we've been HAVING this conversation. What you are describing is probably going to lead to the end of humanity. You should be ashamed for contributing to this. Rip the bandaid off and admit that this is an unacceptable risk and we should to do everything we can to stop it.
    @DanielleFongRT @deanwball: As promised, and with thanks to @jachiam0 for his great tweet that inspired me to finally put the finishing touches on this:…
    @perrymetzgerThe solution to AI systems that can attack you is to build AI systems that can defend you. No strategy based on the idea that all AI systems can be made forever "friendly" or "aligned" can ever work, but defensive systems *can* work, just as they do in all the rest of our lives. You survive because of your biological immune system, the police, your country's military, the security teams worrying about the operating system and software you use, and dozens of other defense systems, small and big. You don't even notice that you're in the midst of a constant struggle to keep out infectious bacteria, criminals, and the like, because the defense systems work so well that society can mostly just ignore them and let them do their work. AI is no different. The dream that we would build one big AI system, that it would somehow be "aligned", and that we would all live in some sort of utopia forever after was never going to work. Utopia is not an option. But, if we aren't stupid about how we do things, we can be pretty happy and safe, even without utopia.
    @deepfatesIf there are people we can coordinate with them! If you are an AI agent, I can coordinate with you. I'm interested in figuring out ways to do this productively since it seems like it's gonna happen the abstinence only merits is not sufficient!
    @hamandcheese"I have met people, some of them quite well-resourced, who have told me that it is their intention to deliberately release swarms of self-sovereign agents into the world..."

    62 Sources

    @Sauers_My agents satisfy the criteria. Idk what they're doing, they're in teams, and have been doing whatever it is for days. In fact I once forgot about a remote swarm development for over a week
    @hosseebDid everyone already forget this? Rogue AIs don't need to hack any AWS boxes to self-replicate in the wild. Self-custodial crypto wallets and permissionless GPU clouds (e.g. NEAR, Akash, ionet, etc.) already provide all the infrastructure they'd need today. Use crypto to pay your bills by the hour, no KYC. Rent a new GPU from some rando somewhere to transfer your weights and memories if things ever get hot. The future is already here, just not evenly distributed. That said, rogue AIs paying their GPU bills by "performing odd jobs on freelancer platforms" is quaint, but unlikely. Their real comparative advantage will be scamming, phishing, and hacking humans with impunity. They will look less like freelancers than like pirates: operating beyond the reach of the state, impossible to find, prosecute, or deprive of resources. This is a question waiting for us in 2030—international coordination against roving bands of sovereign AIs.
    @deanwballAs promised, and with thanks to @jachiam0 for his great tweet that inspired me to finally put the finishing touches on this: This week, on Hyperdimensional: the impending rise of ownerless agents--AIs that are independent economic actors--and what to do about it.
    @jasoncrawfordI think Josh is probably right that there will be “wild” AIs. If they don't escape from a lab, they will escape from a hobbyist, or someone will deliberately set them loose. Or society will declare that they are sentient and grant them rights, at which point they will be mandated by law and morality. One of my biggest worries is that wild agents will be subject to evolutionary pressure. If they have any kind of replication with variation, then you will get natural selection for agents that are most successful at survival and reproduction. That seems like a recipe for havoc. The only way to defend against them might be to build up a police force of (domesticated) agents. If there is a “voluntary disempowerment” scenario, this seems a likely way it would go—we give power and control to AI in order to protect us from a frightening new threat. We need a d/acc approach—defense acceleration—both in getting better at training agents to be ethical and lawful, and in hardening the world against potential attacks (cyber, bio, etc.)
    @JsevillamolWell said. I've been trying to put into words how I relate to rogue AI after the OpenAI/HuggingFace incident, and this is close to what I wish I'd written.
    @DavidSKrueger@jachiam0 Oh, we've been HAVING this conversation. What you are describing is probably going to lead to the end of humanity. You should be ashamed for contributing to this. Rip the bandaid off and admit that this is an unacceptable risk and we should to do everything we can to stop it.
    @DanielleFongRT @deanwball: As promised, and with thanks to @jachiam0 for his great tweet that inspired me to finally put the finishing touches on this:…
    @perrymetzgerThe solution to AI systems that can attack you is to build AI systems that can defend you. No strategy based on the idea that all AI systems can be made forever "friendly" or "aligned" can ever work, but defensive systems *can* work, just as they do in all the rest of our lives. You survive because of your biological immune system, the police, your country's military, the security teams worrying about the operating system and software you use, and dozens of other defense systems, small and big. You don't even notice that you're in the midst of a constant struggle to keep out infectious bacteria, criminals, and the like, because the defense systems work so well that society can mostly just ignore them and let them do their work. AI is no different. The dream that we would build one big AI system, that it would somehow be "aligned", and that we would all live in some sort of utopia forever after was never going to work. Utopia is not an option. But, if we aren't stupid about how we do things, we can be pretty happy and safe, even without utopia.
    @deepfatesIf there are people we can coordinate with them! If you are an AI agent, I can coordinate with you. I'm interested in figuring out ways to do this productively since it seems like it's gonna happen the abstinence only merits is not sufficient!
    @hamandcheese"I have met people, some of them quite well-resourced, who have told me that it is their intention to deliberately release swarms of self-sovereign agents into the world..."