Investor Ramez Naam warns concentrated AGI power poses severe societal risks
If AGI is achievable & labs can be banned from using a model internally ONLY if they release the model publicly, the Big Three labs may decide it is better to capture all the value from AGI themselves by expansion & acquisition. Sharing AI access with other firms triggers risk.
Many users criticized big AI labs hoarding AGI internally to dodge regulatory bans, warning it would create closed empires, stifle competition and open-source work, and concentrate power and value in harmful ways.
Seems like a bad incentive for the rest of the economy, and for workers.
If AGI is achievable & labs can be banned from using a model internally ONLY if they release the model publicly, the Big Three labs may decide it is better to capture all the value from AGI themselves by expansion & acquisition. Sharing AI access with other firms triggers risk.
This would not be a desirable outcome. The greatest AI danger comes from concentration of power. Democratizing and spreading access to AI does come with risks, but overall is likely to be less dangerous and bring more benefit to society than AI in the hands of the very few.
If AGI is achievable & labs can be banned from using a model internally ONLY if they release the model publicly, the Big Three labs may decide it is better to capture all the value from AGI themselves by expansion & acquisition. Sharing AI access with other firms triggers risk.
I never thought about this, but yeah, if you don't release there's no mechanism or rules that say you have to report anything you have discovered. Not yet anyway. So there's only economic and benchmark pressure. If you padded your lead enough you could release incremental upgrades while advancing the frontier invisibly internally. The main problem with this would be compute constraints I guess. Other than that this would probably be the preferred route for some labs.
If AGI is achievable & labs can be banned from using a model internally ONLY if they release the model publicly, the Big Three labs may decide it is better to capture all the value from AGI themselves by expansion & acquisition. Sharing AI access with other firms triggers risk.
@emollick Coming up w sweeping AI policy on the fly is not a going to end well. Has any third party come up w a reasonable regulation stance that people can start rallying behind?
@AndrewCurran_ @emollick @akatzzzzz Yes, it’s bizarre. Foreign nationals are apparently banned from working on Mythos/Fable within Anthropic, but can look at whatever internally is beyond it
@emollick They can kiss their exemption from copyright violations goodbye imo if this is the case.
@emollick If the above proves true then the emergence of techno giga-corps is very likely
@QuentinWKramer @emollick Why?
@emollick this was always going to be the way in the endgame
@emollick Big Three labs' vertical integration might stifle open-source workflows as proprietary API latency and cost become the primary competitive moats.
@emollick they don't offer the models out of charity now. You make it seem like they would be better off economically keeping the models internally, but there is a reason they release them.
Besides the immediate cash flow, I think it's actually quite hard to capture value downstream.
@emollick underexplored angle
@AndrewCurran_ @emollick @akatzzzzz I suspect the definition of "release" may get fuzzier and fuzzier too
@emollick and if they go the internal-only route, regulators just write new laws around concentration risk
curious if the gap creates its own resistance
@emollick Yea especially if your partial owners rat on you in the process
@emollick Quand l'innovation se concentre, la valeur et le pouvoir se concentrent aussi. Sans gouvernance claire, l'AGI risque de devenir une rente pour quelques acteurs, et non un progrès partagé.
@emollick Regulation that punishes sharing access will simply accelerate the creation of closed AI empires. The opposite of the intended effect.
@emollick Are you suggesting MythosV2 already exist? Hence the impact is a lot less severe then what people believe to be? Meaning Anthropic has no incentive to rush out Fable/Mythos Back out? :)
@AndrewCurran_ @emollick @akatzzzzz So basically Ilya 😆😆😆😆
@emollick Internal bans without release would kill competition