Researchers Cut LLM Reliability Evaluation Costs By Up To 156x

As LLMs saturate benchmarks, evaluating their five-nines reliability is crucial, but prohibitively expensive. We cut the inference cost by 5-20x on average (up to 156×) by exploiting a key insight: LLM failures are not random. 🧵[1/n]