Samuel Albanie Notes Big Jump in AI Math Time Horizon

Samuel Albanie, frontier evals lead for Gemini at Google DeepMind, shared a tweet stating there has been quite a big jump in the no-CoT time horizon. The attached chart is titled Figure 41 and plots the 50 percent reliability time horizon on mathematics competition problems against public release date. It displays a 95 percent bootstrap confidence interval for Astra. The post comes from his account on X and includes the chart as an image attachment.