What are the best examples of an LLM API's behavior changed despite the LLM name/version being exactly the same?
I'd like to collect as many examples of these as possible.
8:41 AM · Jul 5, 2026 · 3.9K Views
Sentiment
Users expressed frustration with model drift in fixed LLM API versions, describing it as a headache for developers.
Pos
0.0%
Neg
100.0%
1 comments with sentiment.
Cluster Engagement
Digg Deeper
No Digg Deeper questions have been answered for this story yet.
Related links
Posts from X
Most Activity
Most Activity
VIEWS645BOOKMARKS1LIKES7
Graham Neubig@gneubig
I know this Anthropic blog about serving issues causing accuracy to go down for instance: https://www.anthropic.com/engineering/a-postmortem-of-three-recent-issues
Graham Neubig@gneubig
What are the best examples of an LLM API's behavior changed despite the LLM name/version being exactly the same?
I'd like to collect as many examples of these as possible.
4hViews 645Likes 7Bookmarks 1
Strata@ChainZenit
@gneubig man, models drifting over time is such a headache for devs.
4hViews 31
Lunari@0x_lun
@gneubig the openai gpt 3.5 turbo situation where json reliability silently tanked for weeks is the classic one
system prompt following also drifts without any version bump and almost nobody catches it until evals break
4hViews 20
Deepak Vijaykeerthy@deepakvijayke
@gneubig Does this count?
4hViews 13