Commentator @tenobrus argues perceived AI degradation is a psychological illusion driven by hyper-vigilance after inevitable errors

@tenobrus it was weird being on fable for a week, and it really felt like "it just worked" on everything I threw at it. now i'm back on Opus 4.8 and it's yelling at me all the time, giving weird outputs and just feels like such a downgrade.

@tenobrus this is why the fable ban is genius marketing. give everyone a taste, then snatch it away while the memory is still rosy

@ryiacy just looks like noise to me man

@tenobrus I think it's true that the initial model shine wears off but it definitely is also true that model performance is bumpy in a measurable way. Looking at http://marginlab.ai, Anthropic's performance on public evals goes up and down a lot

@tenobrus it's funny that I mostly don't get this! takes at most an hour to see a new model make a dumb mistake just like all the previous ones, like ah right it's still an llm it still does all the usual llm things

@tenobrus another hypothesis is the models are promoted to incompetence.

People sic the model on harder and harder tasks until it reaches a level of incompetence, from where the user then complains on twitter, and every frontier model appears to be frustrating

@tenobrus will definitely apply to fable cause no way that high of an esteem can last. but it will apply less and less as models get better. and maybe in the limit the effect reverses and you get more and more amazed before normalizing

@tenobrus I think it’s as simple as people increasing complexity until the model is garbage

@tenobrus As time moves forward, anthropic has more chance to slop code their API and Claude Code and cause new imaginative regressions

@tenobrus yes, my theory:

> familiarity. at first you prompt longer and more thoughtful > variability. you remember the good at first, then you get used to it and notice the bad > they really do degrade. quants, load balancing stuff, exploits mean system message changes means regression

@tenobrus This happens with people too. It’s called the honeymoon phase.

@tenobrus imo it's the opposite: they use the new model and it feels like a clear upgrade, so they start trusting it more, too quickly. as soon as the cracks show, they're convinced it must be degraded rather than user error in expecting too much. that and the usual negative polarization

@tenobrus uhm not to out myself as a broken husk of a man but...is anybody else totally familiar with this...because it also happens with new people...they turn out to be disappointing but in a new and maybe more advanced way...? (please someone say yes please)

@tenobrus For me Opus 4.6 Thinking is still the best for Brainstorming and steering other models, I don’t know how else to describe it, but while it isn’t as performative as a tool it’s still much better at maintaining multi-layer state awareness in a way that other models don’t

@tenobrus The randomized nature of LLMs make them perfect target for all sorts of biases

@iwasborngood Sometimes. But, not in every instance. OAI basically radically changed its model performance because people were falling in love with the models and the older models played into it hard. Anthropic's first rounds of degradation were compute constraints. But, that's not all of it.

@repligate @tenobrus but if we assume that models are conscious beings, would not it be logical then that they can be tired, lazy, inattentive? just variational in performance basically

@tenobrus Beyond the practical starting to notice the flaws, I think there may be personality exhaustion. The models have their habitual tics that can grind, areas it mentally flows badly with ("You are absolute right-" except all the time in minor ways) and new models are altered