The case for starting AI evaluations small
One post argues that a few good tasks and verifiers are enough to get started, provided you can analyze results and use them to improve your evaluations.
TLDR
“The cold start is smaller than people think,” a user argues about building evaluations. Instead of designing a comprehensive set of tasks and verifiers upfront, they recommend starting with a few good ones, analyzing outcomes and traces, and using those findings to update and expand the evaluations.
Combined views
25.5K
2 Sources, first seen ago
106 likes4 comments147 saves7 reposts
