The critique followed reports of query nerfing on Fable
Prompted by the Fable "nerfing on frontier AI development related queries" stuff but the point is more general...
I have criticized OAI many times for silent A/B testing, which I think is inappropriate for such a critical technology
I tentatively think that silent model switching is never a good idea.
It's horrible for research (including safety research), among many other effects
That doesn't mean Ant + others should just sit there and tolerate abuse.
There is a large action space, including throttling + issuing warnings, investigating the abuse, etc.
Prompted by the Fable "nerfing on frontier AI development related queries" stuff but the point is more general...
I have criticized OAI many times for silent A/B testing, which I think is inappropriate for such a critical technology
@BlackHC @yong_zhengxin Not sure I follow what point you're trying to make. Sounded like you were defending the [model/system/whatever] switching thing, but now I am not sure
@Miles_Brundage @yong_zhengxin I guess that's why Fable and Mythos are separate offerings because one can simply view Fable as the whole system (incl steering vectors etc)? Obv this won't allow valid inferences for Mythos
It also means you don't get a feedback signal on false positives - people can't complain if they don't know it's happening.
@BlackHC @yong_zhengxin One might choose to call it something other than model-switching (PET, steering vectors... sounds like effectively model switching to me, but anyway)... point is, it is a silent degradation
@Miles_Brundage @yong_zhengxin It is not switching though. Still using Fable but sandbagging via prompt injection?
@Miles_Brundage @yong_zhengxin I guess that's why Fable and Mythos are separate offerings because one can simply view Fable as the whole system (incl steering vectors etc)? Obv this won't allow valid inferences for Mythos
@BlackHC @yong_zhengxin One might choose to call it something other than model-switching (PET, steering vectors... sounds like effectively model switching to me, but anyway)... point is, it is a silent degradation
The critique followed reports of query nerfing on Fable
Prompted by the Fable "nerfing on frontier AI development related queries" stuff but the point is more general...
I have criticized OAI many times for silent A/B testing, which I think is inappropriate for such a critical technology
I tentatively think that silent model switching is never a good idea.
It's horrible for research (including safety research), among many other effects