Older LLM benchmark scores may be less useful for predicting future performance

A researcher argues that LLM evaluation should account for time: models are trained on past data but deployed later. On LiveCodeBench, they say using 12-month-old rather than one-month-old data to predict next month’s scores more than doubled prediction error. They report that Chronos-2 cut next-year prediction error by 37% compared with pooling past benchmark scores across 19 models and 12 datasets, and recommend weighting recent data more heavily.