Lorenzo Pacchiardi leads a paper arguing that conventional AI evaluation methods based on static models are structurally inadequate for continual learning systems and proposes recentering on behavioral trajectories
Reposted by Gavin Leech, it was discussed among AI safety accounts.
#1457Seán Ó hÉigeartaigh@S_OHEIGEARTAIGHDelighted to have coauthored this paper as part of a great team led by @LPacchiardi . What happens if we get continual learning to actually work in frontier AI models? Much of our current governance is based on periodic evaluation of static models. Such governance will break. We propose a direction for addressing this.
Yes, it's uncertain whether continual learning in frontier AI will be achieved, even if company leaders like Amodei are confident. But the evaluation and governance communities are struggling to keep up with the pace of change; we need to change that and start planning not just for what's here now, but what the research community is targeting as goals. Skate to where it looks like the puck is going, not where it is now.
(When I'm back in the autumn, I might work with the team on a more governance-focused companion piece).
Lorenzo Pacchiardi@LPacchiardi🚨 New paper: AI evaluation is structurally unsuitable for continual learning (CL). To address this, evaluation should be centred on the "behavioural trajectories" that CL systems develop, with the goals of characterising possible behaviours and forecasting their evolution. 🧵