Experts Roast AI Evals for Creator Skills and Demo Error Discovery in Claude
Reactions from ranked influencers
3 posts
In my next episode, I asked @sh_reya and @HamelHusain (taught evals to 4,500+ engineers and PMs) to roast the AI evals that I built for my creator skills live. They also showed me how anyone can use their free Error Discovery skill in Claude Code or Codex to turn real AI…
“The fundamentals for [AI evals] still apply. Start by looking at real data. What has changed is getting agents to help you look at it in a thoughtful way.” Here’s my new episode with @sh_reya and @HamelHusain, who have taught AI evals to 4,500+ engineers and PMs. I asked them…
There are two types of AI evals - tops down and bottoms up. From @sh_reya: “Think of top-down as: If you’re in a vacuum, just given the task description, what would you come up with? Claude does a very good job of helping you with these top-down evals. Bottom-up evals are the…
“The fundamentals for [AI evals] still apply. Start by looking at real data. What has changed is getting agents to help you look at it in a thoughtful way.” Here’s my new episode with @sh_reya and @HamelHusain, who have taught AI evals to 4,500+ engineers and PMs. I asked them…