Critic Accuses Artificial Analysis of Chart Crimes
Teortaxes publicly flags alleged chart errors in Artificial Analysis AI model benchmarks.
Teortaxes posted a critique of charts by Artificial Analysis, pointing to alleged errors in AI model benchmarks. Artificial Analysis replied that there had been a temporary error with cache hit rate calculation, which was fixed minutes after the screenshot. The company stated that the model shows only marginal differences in cost per task and remains on its Pareto frontier for intelligence versus cost when accessed via the DeepSeek API with the applicable discount. The post and response form a direct exchange over the presentation of benchmark data.
Amazing innovation in chart crimes here @ArtificialAnlys I get that Sama needs his victory lap, many fools asked me "how will DeepSeek recover after Luna price cuts??!", but… Sam is strong. He can take it. Fix.
@teortaxesTex @ArtificialAnlys it uses as many tokens as the previous version, so i suspect something went wrong here. so i suspect that v4 flash is 50 @ 0.02, which will break all curves
Reactions from ranked influencers
1 postTemporary error with cache hit rate calculation - it was rectified a couple of minutes after your screenshot! 0731 is only marginally higher Cost per Task than the earlier version, and via the DeepSeek API with ~99% cache hit discount it is most certainly on our Pareto…
