The case for multiple independent AI evaluators

Gary Marcus credits @DKokotajlo with the point that AI evaluation can use more than one independent evaluator, and argues that it should. He says a mix could balance expertise in human failures, cybersecurity and model capabilities, along with different ideologies and perspectives. He compares the approach to grant panels that bring together scholars with different specialties.