Developer Aaron Gokaslan and systems engineer Yacine warn that silent AI model updates are injecting bugs into distributed code
Silent updates also cause errors in model card queries.
Users criticized AI models for silently nerfing capabilities without notice, calling refusals on topics like math and mitochondria absolutely reckless.
Most Activity
@difficultyang There are some examples going around Twitter, it won’t even answer questions about its own model card correctly and will inject subtle bugs in distributed code it seems.
@SkyLi0n Inability to answer about the model card was straightforward classifier downgrading you to Opus. I also am able to trigger this. I am looking specifically for evals where Fable /should/ be good at something but it isn't.
@yacineMTB @karpathy Especially when you see this thing refuse to answer questions about math and mitochondria…. Absolutely reckless to not give the user a heads up
@yacineMTB @karpathy Mythos without safeguards
Who’s building this?
@SkyLi0n @difficultyang > inject subtle bugs in distributed code do you have an example?