@emollick Not only have they not succeeded explaining it, but they have no response to what are obvious false positives, like nerfing the model for eg the Semi Analysis team and many others.

It sits poorly for paying customers when their vendor silently and deliberately nerfs the model

@emollick The thing is, you can't tell people "I know better than you.". And you need to be sincere. Which means you need to sincerely believe that you don't know better than them.

Yes

@emollick This is why, even though Anthropic has the best product, and the best team, I'm bearish on them long-term. Arrogance eventually ruins everything.

@emollick We’ve reached the point where an institution/firm can publish 100s of pages of reasoning and transparency, and many people will still respond to the presumed motive instead of the argument.

@zooko @emollick I think Dario is clearly scared of the possibility, even though he's not resigned to it. But I don't think that's what drove any of this mistake, just inexperience. (Altman is too political for me to think that I can infer anything about his beliefs from what he says.)

@emollick A sincere fear that biology as a whole discipline could be misused is paranoia, releasing it with the worst solution possible is profitable paranoia.

@emollick There’s currently a strain of Ebola out there with no vaccine. Why don’t they have the mythical fable make one? There have been vaccines for other strains of Ebola and nobody has made a super virus, so it wouldn’t prove anything if they did it but come on. Make that vaccine.

@zooko @emollick I would settle for: they *suspect* they know better than everyone, but acknowledge to themselves (forever) that they are not *sure*.

@emollick Because we know that if the situation was flipped, and American models were 6-12 mo behind Chinese ones, the same people would be screaming bloody murder if DeepSeek added anti-distalliation filters.

@emollick The bad part of the safeguards in (1) is they help advantage attackers in breaking stuff (economy of scale for jailbreaks), and disadvantage defenders in protecting stuff.

It'd be different if it was "safeguards on security prompts get reduced at XYZ time 48 hours from now)"

@emollick One thing is true: (1) You can’t speak for the truth of the intentions of the minds of other people.

@davidmanheim @emollick Hm... interesting that we seem to be talking past each other! I assume that the eternal truths about evolution, economics, cooperation, security, social organization, and ethics would still be true in a world populated solely by AIs. :-)

@davidmanheim @emollick If Anthropic and/or OpenAI think those things are going to stop mattering, then that's another reason that I'll be suspicious and bearish about them. :-)

@davidmanheim @emollick And, yeah, if Amodei or Altman think of themselves as short-termers who won't be held accountable for their work, then that would make me trust them less.

@davidmanheim @emollick Yes, I don't expect these basic truths about virtue and social organization to change.

@emollick It seems mostly marketing from Anthropic. And they really want to have a Monopoly on AI.

@davidmanheim @emollick Like… those truths weren't handed down by God, and they weren't invented by a philosopher. They arise out of physics, by way of biology, etc. Why would they stop arising?

@davidmanheim @emollick Huh. I hold that prospect with low confidence, at a similar level as the prospect that a given CEO or leader will die or be replaced by another human.

@zooko @emollick I think it's true of OpenAI as well, and I would agree that in the long run it's usually ruinous, but don't think that there's enough of a long run for the mistakes to return home to roost. Do you think that there's more than a decade left before this stops mattering?