• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Self-promo

    Viktor is pitched as a Slack assistant that reviews AI-agent evals overnight

    In a partnered post, a researcher says Viktor checks failed task logs and suggests harness changes for human review.

    elvisEL
    1 Source, 53m ago, first seen 53m ago

    TLDR

    A researcher who runs nightly agent-harness experiments says reading the results was taking up his mornings. In a partnered post, he describes Viktor reviewing results overnight in Slack. In his hypothetical example, Viktor checks the logs for 23 tasks that stopped passing, traces the failures to one change and suggests undoing it. The researcher says he checks the logs and makes the final call.

    Combined views

    2.2K

    1 Source, first seen 53m ago

    Combined views

    2.2K

    1 Source, first seen 53m ago

    16 likes
    16 likes
    9 comments
    7 saves
    1 reposts
    9 comments
    7 saves
    1 reposts
    Featured Source

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    elvis@omarsar0Reading eval results is now the slowest part of building agents. I'm Elvis, founder of @dair_ai. I lead research, build, and teach about AI agents. I run harness experiments every night, but reading the results was eating my mornings. Every change to my harness gets evaluated overnight, whether it touches memory, tool use, or context compaction. The morning after is the hard part. I check which tasks my agent got right yesterday but wrong today. Then I open the logs for each failure, one by one, to figure out which of my changes caused it. I tried a dashboard first. It showed the pass rate dropped. It couldn't tell me why. That is the job Viktor, an AI employee in Slack, is built for. He reviews the results overnight. Here is how that plays out. Say 23 tasks that passed yesterday fail today. Viktor checks all 23 logs, traces them to the one change that caused them, and suggests undoing it. I check the logs and make the call. Viktor does the digging. I decide what goes into the harness. He is also proactive. He flags problems before you ask, which helps you stay on track with complex eval runs and other research tasks. Harness engineers, do you check every eval run, or only when the pass rate drops? Try free at @viktor_com. $100 in credits, no card. Full link in my first reply. Thanks to the team for partnering with me on this post2h

    1 Source

    elvis@omarsar0Reading eval results is now the slowest part of building agents. I'm Elvis, founder of @dair_ai. I lead research, build, and teach about AI agents. I run harness experiments every night, but reading the results was eating my mornings. Every change to my harness gets evaluated overnight, whether it touches memory, tool use, or context compaction. The morning after is the hard part. I check which tasks my agent got right yesterday but wrong today. Then I open the logs for each failure, one by one, to figure out which of my changes caused it. I tried a dashboard first. It showed the pass rate dropped. It couldn't tell me why. That is the job Viktor, an AI employee in Slack, is built for. He reviews the results overnight. Here is how that plays out. Say 23 tasks that passed yesterday fail today. Viktor checks all 23 logs, traces them to the one change that caused them, and suggests undoing it. I check the logs and make the call. Viktor does the digging. I decide what goes into the harness. He is also proactive. He flags problems before you ask, which helps you stay on track with complex eval runs and other research tasks. Harness engineers, do you check every eval run, or only when the pass rate drops? Try free at @viktor_com. $100 in credits, no card. Full link in my first reply. Thanks to the team for partnering with me on this post2h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet