• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Reaction

    The limits of counting human touches per pull request to evaluate coding agents

    A user says human touches per pull request are easy to track, while scoring a PR is difficult.

    SR
    1 Source, 2h ago, first seen 2h ago

    TLDR

    A user says they use human touches per pull request to evaluate coding agents because the count is easier to track than a PR score. They caution that more touches aren't necessarily bad on a complex task, even when an agent is doing well.

    Combined views

    2.4K

    1 Source, first seen 2h ago

    10 likes

    Combined views

    2.4K

    1 Source, first seen 2h ago

    10 likes
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    4 comments
    1 saves
    1 reposts
    Featured Source
    4 comments
    1 saves
    1 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    @sydneyrunkleit's really hard to "score" a PR but "human touches per PR" is quite easy to track one complicating factor here though is even if anagent is doing well, on a complex task, more touches isn't necessarily bad we've been using this metric for evaluating our coding agents!

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    @sydneyrunkleit's really hard to "score" a PR but "human touches per PR" is quite easy to track one complicating factor here though is even if anagent is doing well, on a complex task, more touches isn't necessarily bad we've been using this metric for evaluating our coding agents!