AI eval experience reportedly sought in nearly half of 25 product management openings

Every, citing 25 product management openings shared by @Lennysan, says nearly half sought experience writing tests of AI performance on specific tasks. It says it helps people build benchmarks from their own work and standards to decide whether to switch when a new model launches. When several models can do the job, Every points to speed, cost and how closely the output matches what you'd have written; it says a leaderboard won't tell you whether to switch.