An internal AI intelligence measure comes with a model-bias warning

The poster says they dislike public benchmarks but their team is experimenting with how to communicate its internal measure of AI intelligence. They caution that it is likely biased toward their own models and suggest focusing on the relative ranking of open models. They say they excluded hosted models because of how those services handle data and to keep their validation set secure.