Loading activity…
@vincentsunnchen
research & founding team @SnorkelAI. previously, @StanfordAILab, @hazyresearch.
AI
Snorkel AI's announcement says a new red team will continually test benchmarks and a fellowship will support independent contributors.
AI
Founder Marissa Mayer says she'll discuss Dazzle on MTS and sums up its approach as “Pictures, not prompts!”
AI
The host of a founder dinner with a Snorkel AI participant said a challenge for reinforcement-learning environments is telling whether a model’s failure stems from the input, test harness or reward model.
AI
One post urges AI product builders to spend more than 25% of their time creating benchmarks and getting model labs to care. A quote post argues that strong benchmarks also need a clear purpose and rigorous quality checks.
AI
A post argues that data, once viewed as “plumbing,” can now openly be described as the research itself—even at frontier labs.
AI
A post makes the case for “eval pluralism”: more benchmarks, input from clinicians, engineers, scientists and everyday users, and open infrastructure for evaluating AI.
Technology
Scientists from labs worldwide shared their actual research workflows to test AI agents. The result? Even top models solve just 30 percent of these authentic tasks.
