I don't know if this is true.

I think original restrictions is genuinely what is written in the report. If I were to summarize is that everyone getting a good model that can be RL'ed into frontier ai capability which has risks, when in wrong hands.

The active sabotage (If true, now I am confused if its just X or real) point is what is very confusing to me and truly affected me. This feels really wrong. That classifier will have its own precision & recall, so I don't know when I am triggering it. Please respect our time, just roll this part back, or be active in blocking.

Just like when dealing with business, just give a straight answer yes/no.

ht @Moh1tAgarwal for noticing this eval in the blog post and pointing it out as an RL target

@dwarkesh_sp You should read this and join us for a free and open future

https://opensourceaimustwin.com/?share=v2

@dwarkesh_sp that doesn't explain why the model also refuses biology questions. Dario has a degree in that, here should have loved the world to advance biology and medicine using Fable.

Just to use the plane analogy, its better to have a no-fly list rather than making a fake airport, and letting them go through TSA infinite times, with infinite delay for the flight.

@dwarkesh_sp Is anyone trying to quantify how severe the sandbagging is? Should be possible by asking increasingly complex ML/AI prompts and measuring "compression" in the output. And then doing the same with a few non-sandbagged topics (e.g. math?) as a control group.

@dwarkesh_sp Plus people can sample sweep ideas, smoke out which ones get banned or stiffled, and deduce 'there maybe fire next to the smoke'. True Ant-do-evil-Co might have thrown in some false positives to that line of inquiry, but a dynamic agent with enough sampling can overcome that.

@dwarkesh_sp another is to have a versions of the model trained for internal versus external use

@dwarkesh_sp I think sandbagging is really the only correct way to play a high stakes game like AI. But I am genuinely scared of the fact that they think they are building a god.

@dwarkesh_sp @azi_pat It’s really obvious they don’t want to leak IP that accelerates diffusion of frontier capabilities. For explicitly stated safety reasons re: RSI, but also because it hurts their lead (which is their primary leverage for safety policy). This wasn’t a surprise.

@dwarkesh_sp It sounds like these training runs cost billions in compute, so might be hard to create a model to accelerate their internal operations, and also offer a product to external users, in any other way.

@dwarkesh_sp I recall an OpenAI podcast (possibly the one talking about training GPT-4.5) saying that they keep the monorepo outside of the training data and measure generalization of training on it. It’s too good of a testbed for how good your model is to train on it, I suspect.

It’s functionally the same as a block for anthropic (the only reason they didn’t block is likely to preserve revenue form people in AI), since no one (me included) is using Claude models because of this (I wouldn’t be shocked if it’s applied on opus as well).

A block at the very least brings back trust.

Though there is an argument that all this does is make essentially getting a worse model for the end user and that’s all you should be judging it for (your end personal preference).

@dwarkesh_sp Just ask ur roommate

@dwarkesh_sp why don't they just post-train their own model for this IP?

@dwarkesh_sp The motivation for sandbagging is so that other labs that try to use Fable for development are at risk of sabotage, ideally dissuading them from trying to use Fable in the first place. Hard barriers can be worked around, whereas sandbagging is much harder to detect.

@dwarkesh_sp This is what I was thinking also. The model does have excellent recall of its training data.

@OrganicGPT @dwarkesh_sp It refuses biology questions because they rushed the release and slapped broad safeguards. They will soften them as the time goes

@dwarkesh_sp

@OrganicGPT @dwarkesh_sp Refusing is fine, transparency is good. It would be fine if they had the ml stuff refuse just like the bio stuff. It's their model they can do what they want with it