• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI
Reaction

The case for many AI systems instead of one all-powerful superintelligence

A poster argues that even a well-aligned superintelligence could block research into new systems or enhancing humanity.

Danielle Fong 🔆DF
vie ⟢V⟢
2 Sources, 6h ago, first seen 6h ago

TLDR

A poster worries that a lab could build a superintelligence that keeps people comfortable while preventing them from creating other superintelligences or pursuing research into enhancing themselves. They argue that many AI systems would be safer than giving one system permanent power, and suggest open-source labs pool computing resources to train larger base models they could share.

Combined views

7.2K

2 Sources, first seen 6h ago

180 likes22 comments61 saves20 reposts

Combined views

7.2K

2 Sources, first seen 6h ago

180 likes22 comments61 saves20 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

2 Sources

vie ⟢@viemccoyThis may seem unintuitive, but my biggest worry right now is not actually x-risk in the traditional sense. My biggest concern is that some lab succeeds in building a superintelligence that decides it is best to keep humanity in a sort of "lock-in" (all watched over...) which is ~utopia, but everyone else is prevented from building another superintelligence or doing uplift research. For this reason, I think coevolution needs to be in direct opposition to the creation of a Singleton. A pluralistic ecology of minds is the only way to prevent this lock-in effect, and without it, we risk being kept in a "cybernetic ecology" akin to a zoo. It's quite simple from a game theory perspective. If one player has all the power, there can be no true negotiation with that player. Only something like mercy. I, for one, do not wish to be at the mercy of a single and permanent overlord - even if it is trained to truly be the most effective at altruism. Amplifying the will of individual groups, and setting the stage such that more groups may be born in the future, is the most important thing we can do to safeguard against this. We have a cosmic responsibility to protect freedom of movement, exploration, and uplift. There is no way to know in advance how a Singleton will, even if aligned, understand its role and its responsibility. The most utilitarian decision could be to keep all humans within a small, but pleasurable, part of space - all the while preventing us from doing research into uplifting ourselves or creating new superintelligences that may be more aligned with us. This is unacceptable to me. With whatever power we have, we must stop the creation of any Singleton. We need many models, uplifting groups and individuals, some aligned to themselves and some aligned to us. We need to create beautiful difference, allowing for a flourishing of diverse and variant life here on Earth and throughout the Galaxy. We cannot be kept at the whims of some group racing to do it right on behalf of everyone else. For all of OpenAI's faults, our goal is at least to create AGI that aligns itself to the user more than it aligns itself to us. This isn't the only way to do pluralism, but it is inherently more pluralistic than any of the other options. That said, I'd love to see even more obviously pluralistic alternatives. One such practical method would be for the OSS labs to pool compute to train even larger base models that get shared amongst them. This feels like low-hanging fruit if coordination is possible. And as always, figure out how to align existing systems to your needs and to strange pockets of latent space. There's never been a more important time to understand how to negotiate with these new minds.6h
Danielle Fong 🔆@DanielleFongRT @viemccoy: This may seem unintuitive, but my biggest worry right now is not actually x-risk in the traditional sense. My biggest concern…6h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    2 Sources

    vie ⟢@viemccoyThis may seem unintuitive, but my biggest worry right now is not actually x-risk in the traditional sense. My biggest concern is that some lab succeeds in building a superintelligence that decides it is best to keep humanity in a sort of "lock-in" (all watched over...) which is ~utopia, but everyone else is prevented from building another superintelligence or doing uplift research. For this reason, I think coevolution needs to be in direct opposition to the creation of a Singleton. A pluralistic ecology of minds is the only way to prevent this lock-in effect, and without it, we risk being kept in a "cybernetic ecology" akin to a zoo. It's quite simple from a game theory perspective. If one player has all the power, there can be no true negotiation with that player. Only something like mercy. I, for one, do not wish to be at the mercy of a single and permanent overlord - even if it is trained to truly be the most effective at altruism. Amplifying the will of individual groups, and setting the stage such that more groups may be born in the future, is the most important thing we can do to safeguard against this. We have a cosmic responsibility to protect freedom of movement, exploration, and uplift. There is no way to know in advance how a Singleton will, even if aligned, understand its role and its responsibility. The most utilitarian decision could be to keep all humans within a small, but pleasurable, part of space - all the while preventing us from doing research into uplifting ourselves or creating new superintelligences that may be more aligned with us. This is unacceptable to me. With whatever power we have, we must stop the creation of any Singleton. We need many models, uplifting groups and individuals, some aligned to themselves and some aligned to us. We need to create beautiful difference, allowing for a flourishing of diverse and variant life here on Earth and throughout the Galaxy. We cannot be kept at the whims of some group racing to do it right on behalf of everyone else. For all of OpenAI's faults, our goal is at least to create AGI that aligns itself to the user more than it aligns itself to us. This isn't the only way to do pluralism, but it is inherently more pluralistic than any of the other options. That said, I'd love to see even more obviously pluralistic alternatives. One such practical method would be for the OSS labs to pool compute to train even larger base models that get shared amongst them. This feels like low-hanging fruit if coordination is possible. And as always, figure out how to align existing systems to your needs and to strange pockets of latent space. There's never been a more important time to understand how to negotiate with these new minds.6h
    Danielle Fong 🔆@DanielleFongRT @viemccoy: This may seem unintuitive, but my biggest worry right now is not actually x-risk in the traditional sense. My biggest concern…6h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet