Arvind Narayanan and Peter Norvig argue that benchmark-driven AI evaluation fails to account for future developmental trajectories

@Avameanssong @kalsbskk81826 @sanmikoyejo @philiptorr @yong_suk_lee @joelbot3000 @NorvigPeter @random_walker Its an oral at IMCL

link to the paper: https://openreview.net/pdf/c0833e1fef52e998ef8d65944caaa3aae0eaa35c.pdf

@random_walker cc @Afinetheorem

@random_walker finally someone naming the problem *and* offering a way forward instead of just roasting the whole field

@AmirSariri @random_walker Right on - ours on this coming next week, dude!