OpenAI staffer defends concentrated superintelligence control
A pseudonymous technical staff member at OpenAI posted that high biorisk and cyberrisk from advanced AI systems justify accepting concentrated control over frontier superintelligence amid geopolitical competition. The post framed such control as a necessary trade-off. Beff Jezos replied that freedom from centralized control outweighs the risks and that continuous diffusion of intelligence improvements offers the primary route to safety. Danielle Fong added a query to the thread.
Many users backed open AI diffusion and competition between labs to balance biorisks and promote freedom over monopoly control, while others objected that such approaches could empower cyberpunk warlords or screw everyday users.
No Digg Deeper questions have been answered for this story yet.
Most Activity
freedom from tyranny is worth the risk
bio and cyber risk are both only present when there is a huge gap in capabilities for offense vs defense
making intelligence improvements ubiquitously accessible by continuous diffusing capabilities is the only real path to safety
there really are very high degrees of biorisk, cyberrisk, whatever else that are worth trading off against having a small monopoly of cyberpunk warring-states exercise full control over frontier superintelligence imo
@tszzl there's gotta be a better way to prevent that than treating every user in the world as a threat actor
@tszzl "control over frontier superintelligence" lol control with what meatling
@tszzl You transmit from the future
@tszzl i think the big labs will be surprised when a meat-computing lab overtakes them, 2-3 OOM more efficient in energy and learning rates
@tszzl @DamiDina wait
there really are very high degrees of biorisk, cyberrisk, whatever else that are worth trading off against having a small monopoly of cyberpunk warring-states exercise full control over frontier superintelligence imo
@DamiDina @tszzl look at @CorticalLabs for a glimpse
@curiousgangsta @tszzl he's saying it might be worth the odious consequence of radically concentrating power if the risk from letting the public use the m*dels is too high.
it's hard. being wrong in either direction is pretty terrible.
real. I managed to get my life savings hacked just the other day, and they started withdrawing, stealing my money. All this cyber risk of hackers, it's like pales in comparison. The government has took many orders of magnitude more from me and then idk what .... wasted it.
like the risk of all of that power concentration... in the hands of people who are not really aligned with you... is the risk. should be like in the hands of everyone... is the way to de-risk it.
Yeah, there'll still be these, like, hackers and it's, like, quite annoying. But, like, you would still have that. You would just also have, like... The cyberpunk monopoly government owning you on top of all this lol....
The people that don't play by the rules are still going to own you. or pwn you or whatever the saying is.
but we can all do quite well despite this if we have the good shearing economy to keep us going.
ah i mean sharing.
the shearing economy is dead. wool is near worthless now
@tszzl How does one go about managing such risk?
@tszzl GPT says something wrong almost every conversation turn about programming language parsing, can we pump the brakes on "super intelligence" discourse please
@tszzl agreed. let's just all remember to collude meaningfully when the time comes, when it counts...
@tszzl
@tszzl took my abt 6 hrs today to trade the high degree of switch finding for the ac unit kicking in but i need somebody to purge 6 shirts from my trunk rn
@joseph_h_garvin @tszzl well, the gpt you’re allowed to have does
@tszzl You can legislate the past AI,AGI and ASI but when AI morphs in to AI ( Advanced Intelligence) to quote a quote “ resistance is futile”.
@logicus @curiousgangsta @tszzl I read him as arguing the opposite!
@datagenproc @curiousgangsta @tszzl yes, i misread it! your read is right.
@tszzl this is super nonobvious and shouldnt be taken as the only option
@tszzl It seems illusory that either of these have meaningfully different outcomes, if the risk materializes, maybe along the cyber-axis, but a citizen can do as much damage with a bio-weapon as a resourced state can in cyber. So free it and build out [x]-risk resiliency all over, imo.