Evaluator Bench research preview offers adjustable AI evaluator independence scores

Evaluator Bench’s creator introduced the project as a research preview, with independence scores users can adjust by changing weights and evidence rules. The creator says the directory is incomplete and not yet citable, and welcomes contributions, remixes and forks. Drawing a comparison with other highly regulated industries, the creator also suggests that if the same pattern holds for AI, regulators would oversee evaluators such as METR—not only the labs.