Reactions from ranked influencers
5 posts
Amazing release by the @poolsideai team. I love that they've released the complete trajectories for their evals, that's so cool!
Hoping to increase transparency in evals on model releases. Check out http://trajectories.poolside.ai https://twitter.com/jasoncwarner/status/2079641849191649678
@eisokant love this
@eisokant did you keep default settings for the harbor tasks, esp tb2.1? or increase time / resources (like a lot of others do tbf + i think is valid)
Eval logs are *infinitely* more useful than aggregate scores. Not only can we verify the scores, but we can see how a model succeeds/fails to utilize models’ strengths, allocate tasks well, inform tooling development, etc. Really hope this becomes the standard. Bravo, Poolside!
Hoping to increase transparency in evals on model releases. Check out http://trajectories.poolside.ai https://twitter.com/jasoncwarner/status/2079641849191649678
@eisokant next time i read properly 10/10, no notes. amazing transparency, much appreciated!