• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Peter Yang Describes AI Skill Iteration Pattern

    Asks experts how to update skills for one-shot success after manual fixes.

    HH
    ND
    PY
    5 Sources, 29d ago, first seen 29d ago

    TLDR

    Peter Yang posted about a recurring pattern while using AI skills. He runs a skill that falls short on the first try. He then performs manual iteration with AI until the result is correct. After that he asks the AI how it would revise the skill to achieve one-shot success. Yang identifies as a product lead at Roblox who shares AI tutorials and runs a newsletter. The post poses the question to experts in the replies.

    Combined views

    73.8K

    5 Sources, first seen 29d ago

    Combined views

    73.8K

    5 Sources, first seen 29d ago

    336 likes
    336 likes
    75 comments
    428 saves
    7 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    75 comments
    428 saves
    7 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    5 Sources

    @petergyangWhile I'm cleaning up my AI skills I have a question for the experts here. Sometimes I end up in this pattern: 1. I run my skill and it doesn't get it perfect in one shot 2. I do manual iteration with AI to get it right 3. I then ask AI something like: "How would you update the skill so we can one-shot this?" before reviewing and approving its edits. The problem is AI often overfits on this one thread and over time the skill drifts. Any good solutions here?
    @HamelHusain@petergyang You know what I'm going to say
    @NickADobosWelcome to building evals :) Where you make 1000 cases so it can’t overfit on just one The real art trick is finding ways to write generically, and to think a meta level above. What category of behavior does this fall into? Can you write a rule for that? Imagine you are writing your prompt for public use & 1000s of scenarios If you want to get super advanced, and do this for long trajectory work, you stop writing hard one off rules entirely and instead write stories, values, and culture. Like how CEOs write company values and build employee culture so they don’t need to micromanage. You write a decision making system to handle any scenario.

    5 Sources

    @petergyangWhile I'm cleaning up my AI skills I have a question for the experts here. Sometimes I end up in this pattern: 1. I run my skill and it doesn't get it perfect in one shot 2. I do manual iteration with AI to get it right 3. I then ask AI something like: "How would you update the skill so we can one-shot this?" before reviewing and approving its edits. The problem is AI often overfits on this one thread and over time the skill drifts. Any good solutions here?
    @HamelHusain@petergyang You know what I'm going to say
    @NickADobosWelcome to building evals :) Where you make 1000 cases so it can’t overfit on just one The real art trick is finding ways to write generically, and to think a meta level above. What category of behavior does this fall into? Can you write a rule for that? Imagine you are writing your prompt for public use & 1000s of scenarios If you want to get super advanced, and do this for long trajectory work, you stop writing hard one off rules entirely and instead write stories, values, and culture. Like how CEOs write company values and build employee culture so they don’t need to micromanage. You write a decision making system to handle any scenario.