Evaluator Bench research preview offers adjustable AI evaluator independence scores
Its creator describes a source-linked directory of third-party frontier AI evaluators. Users can change score weights and which evidence counts; the ratings address independence, not quality or competence.
TLDR
Evaluator Bench’s creator introduced the project as a research preview, with independence scores users can adjust by changing weights and evidence rules. The creator says the directory is incomplete and not yet citable, and welcomes contributions, remixes and forks. Drawing a comparison with other highly regulated industries, the creator also suggests that if the same pattern holds for AI, regulators would oversee evaluators such as METR—not only the labs.
Combined views
—
2 Sources, first seen ago
