🎯

@deanwball most of those other things are, properly understood, also objectionable. but don't worry. other people are making those point

Clandestine prompt modification and steering vectors are the sorts of techniques I could see being justifiable for countering distillation attacks, since you don't want the adversarial distiller to know they're being nerfed. But confusingly the model card explicitly states they won't use these techniques for distillation attacks. Left unexplained is why these measures are being applied in secret in the first place.

@thinksmall3r @chammond510 Then why not do it for bio and cyber too?

@deanwball I am more worried about the security aspect. It may refuse to scan for security, but nothing stops them to create zero days for your software that the US government will use to hack later into your systems.

This is a supply risk on steroids.

@deanwball You can deal, because you're someone not working in fields or domains that actually need these things.

Those things are not fine, you just don't need them. There's a difference.

@chammond510 @deanwball Because explicit 'this has been downgraded' messages make it much much easier to get around the restrictions.

Exactly!

The other thing that also rubs me the wrong way is that they see “degradation” as an acceptable mitigation. Service providers should always strive to do their best!

If my lawyer decided to do slop job on my case because of my perfectly legal line of work, I would take that personally!

@deanwball @chammond510 I guess they care more or expect more serious / sophisticated requests regarding this topic than the other ones. The 'shadow ban' thing has been a useful strategy for dealing with adversarial users for a while, you see it in social media, video games, etc.

It's just so widely destructive and trust-destroying, even for people whose world doesn't touch ML at all. I'm a lawyer and now I have 0.5 - 2% fear that if I use Fable to build out some of my systems, the quality of the work will be degraded. Why even risk that? Just be transparent.

@deanwball hopefully can be fixed quickly but the bio safeguards are genuinely inane right now

@deanwball Grok and Gemini were both caught silently modifying prompts in absurd fashion. Do you think OAI does anything of the sort? My guess is something like 80% chance that they do, but they're actually good at it

@deanwball The strongest argument against pausing AI development, and the only one which holds even if AI causes massive damage to the economy and to social order, is the treatment and curing of disease. The situation is not fine.

@deanwball It is a bit puzzling that they did admit to it... what is the point then if it's not even covert?

@deanwball Isn’t the precedent here all of the anti-piracy measures in video games that made them unplayable? I don’t fully support Anthropic’s decision here, but it seems like they are just more overtly enforcing their terms of service?

https://gamicus.fandom.com/wiki/List_of_Games_with_Anti-Piracy_Measures

@deanwball this is the first time Anthropic limits utilization towards something that's broadly accepted as a social good

@chammond510 @deanwball I don’t really understand the trust-destroying part, given that they told us what they’re doing. Are you worried they’ll start throttling capabilities without telling us?

@deanwball @chammond510 Also, I'd guess they think that AI development trigger is less likely to go off for normal, less sticky users

@deanwball Hard to believe you are okay with the second one

@deanwball Many people seems OK with this because it will be used against China, Iran or Russia (something that i find morally wrong).

This will be used indiscriminately globally.