Vals AI launches fellowship offering up to $2,500 weekly to fund new AI benchmarks and evaluations

Many PhD students work on building frontier AI evals.

I strongly encourage them to apply to this fellowship, which would accelerate their research and expand its impact.

Academics and dedicated third-party evaluator organizations co-designing evals could be very synergistic.

Pitch us a benchmark or eval technique. We'll fund you to build it.

We're opening applications for the Vals Fellowship. 3–6 months working on the hardest open problems in AI evaluation, with the resources to actually solve them.

What you get: - Unlimited API credits + budget capacity for GPUs and human data - Vals’ evaluation infrastructure - $1,000–2,500 / week stipend - A network of evals researchers across frontier labs and academia

Location: Both remote / in-person in SF applications will be considered

This is an excellent opportunity to shape how we measure AI! @ValsAI is funding researchers to build the next generation of benchmarks. More details in the thread below ⬇️

Apply here (rolling admission) by June 30th: https://vals-ai.notion.site/3688a877dd8d803e887fef10f0ac4daa?pvs=105

To learn more about the program, see https://www.notion.so/vals-ai/Vals-Fellowship-3668a877dd8d80d6b10ac4bf96c4f6be?source=copy_link

@ValsAI seems like you might need some designers to clean up the claude coded website 🫠

@garybasin @ValsAI nah not a pitch. it looks vibed af. the differentiator to make these tools legible to users and also stand out from competitors is unfortunately through design.

@0xnichy @ValsAI Good pitch

Im trying to build prediction into the goal function of LeWM models. But honestly its turning into micro-models that feed back into a LeWM and... idk its awesome. nobody paying me or supporting my research though. If you want to, and care about LeWM you can. Ill still be here doing it.

@ValsAI open source Remote Labor Index

@stochasticdoggo @ValsAI Only researchers? We are trying to build a benchmark using human data for AI agents

@ValsAI Where do we apply?

@ValsAI TikTok benchmark, I know those two words sound dumb. Having millions of views per month with the least amounts of videos on TikTok means a deep understanding of human psychology

@ValsAI Oh I like this…

@ValsAI Done!!! ✅

@0xnichy @ValsAI I agree with you and I wasn’t being facetious. Vibe design eval would be pretty useful too

@ValsAI what's missing is a clear metric for evaluating ai-driven investment decisions over time

@stochasticdoggo @ValsAI Finance Bench is impressive！

@ValsAI Interesting

@stochasticdoggo @ValsAI hi! pretty cool! can i ask the interview process? please

@stochasticdoggo @ValsAI le real unlock est que cette initiative pourrait finalement révéler les limites actuelles des modèles de pointe en terme de robustesse, et non juste leur capacité à plaire aux humains