Danielle Fong requested technical details on the model's auto-degradation
I tried using Fable to help wrestle a severe medical issue of a close relative that has been stumping doctors for close to two years. The model auto-degrades and the problem remains unsolved.
This safety washing is disgusting. I thought better of anthropic...
Users worry Anthropic's Fable model failure on a medical case will trigger safety classifiers that gatekeep access to useful AI applications.
@sebgehr Can you explain a little more how you detected the auto degradation? I think that this is important
I tried using Fable to help wrestle a severe medical issue of a close relative that has been stumping doctors for close to two years. The model auto-degrades and the problem remains unsolved.
This safety washing is disgusting. I thought better of anthropic...
@spandan_madan I assume it triggered the bio filters since the context includes labs. but anyone building guardrails should be very aware of the false positive issue.
@sebgehr Was the model doing good before it degraded? Curious if this is a regulatory issue or a capability issue.
@sebgehr @dagarfield can you please advocate that these use cases do not end up triggering the safety classifier? I'm concerned this will only be available in a separate gatekept model meant for large institutions
@DanielleFong It says that it switched to opus 4.8 so not that hard to detect. Opus fails at the task