• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Announcement

    Models trained with Harness Learning are claimed to nearly double performance on challenging unseen tasks

    A post describes training a model to revise agent harnesses using execution feedback. It says the model can then improve harnesses for unseen tasks while its weights stay fixed at test time.

    DK
    AZ
    2 Sources, ,

    TLDR

    The author says Harness Learning trains models to revise agent harnesses using execution feedback, then apply that skill to unseen tasks without updating their weights at test time. Models trained this way nearly double their performance on challenging unseen reasoning and multihop question-answering tasks, the author claims.

    Combined views

    3.6K

    2 Sources, first seen 10h ago

    Combined views

    3.6K

    2 Sources, first seen 10h ago

    27 likes
    10h ago
    first seen 10h ago
    27 likes
    2 comments
    19 saves
    12 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source
    2 comments
    19 saves
    12 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    2 Sources

    @ZAlvin391051/8 How do we train a model to adapt to tasks it has never seen, without updating its weights at test time? With Harness Learning, we train a model to revise harnesses using execution feedback. At test time, the model applies this learned skill to improve harnesses on unseen tasks, while model weights stay fixed. Improving agent harnesses takes experimentation. We want a model to learn from those revisions and make better edits on unseen tasks. Models trained with harness learning nearly double their performance on challenging unseen reasoning and multihop question-answering tasks.
    @DanielKhashabiRT @ZAlvin39105: 1/8 How do we train a model to adapt to tasks it has never seen, without updating its weights at test time? With Harness…

    2 Sources

    @ZAlvin391051/8 How do we train a model to adapt to tasks it has never seen, without updating its weights at test time? With Harness Learning, we train a model to revise harnesses using execution feedback. At test time, the model applies this learned skill to improve harnesses on unseen tasks, while model weights stay fixed. Improving agent harnesses takes experimentation. We want a model to learn from those revisions and make better edits on unseen tasks. Models trained with harness learning nearly double their performance on challenging unseen reasoning and multihop question-answering tasks.
    @DanielKhashabiRT @ZAlvin39105: 1/8 How do we train a model to adapt to tasks it has never seen, without updating its weights at test time? With Harness…