• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Niko Grupen Notes Intuitive AI Reasoning Traces at Harvey

    Compares reasoning traces from parametric memory plus study notes to coding agents' outputs.

    NI
    1 Source, 26d ago, first seen 26d ago

    TLDR

    Niko Grupen, Head of Applied Research at Harvey, posted about an interpretability result from work with EngramLab on law firm knowledge. He stated that reasoning traces from parametric memory plus study notes are much more intuitive than the interleaved bash commands produced by coding agents. Grupen indicated this difference affects the UX of long-horizon agents. The post includes an attachment showing a clean UI with a user query.

    Combined views

    12.5K

    1 Source, first seen 26d ago

    Combined views

    12.5K

    1 Source, first seen 26d ago

    64 likes
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    64 likes
    7 comments
    75 saves
    6 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    7 comments
    75 saves
    6 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    @nikogrupenUnderrated interpretability result from our work with @EngramLab on @harvey law firm knowledge that makes a difference for the UX of long-horizon agents. The reasoning traces you get from parametric memory + study notes are much more intuitive than the interleaved bash commands you get from coding agents. When you read them (esp as a non-technical user), you can actually understand what’s going on. For example: "I remember two antitrust matters from earlier work, one that was terminated (client matter 1001-00004) and one that cleared after an HSR Second Request (client matter 1003-00003)" Compare this to the following from a coding agent (1 of 19 such bash commands btw): { "command": "cd /tmp/cache/1003-00003 && echo \"=== engagement letter head ===\" && sed -n '1,25p' Engagement/engagement-letter-hpe-fund-iv.docx.txt | cut -c1-500 && echo && echo \"=== practice group / partner mentions ===\" && grep -rhoi \"[^.]*\\(practice group\\|M&A partner\\|antitrust partner\\|responsible partner\\|lead partner\\)[^.]*\\.\" . | sort -u | head -12 | cut -c1-300 && echo && echo \"=== HSR outcome ===\" && grep -rhoi \"[^.]*\\(waiting period expired\\|expiration of the \\(extended \\)\\?waiting period\\|early termination\\|closed the investigation\\|closing letter\\|consent decree\\|no further action\\|without taking\\)[^.]*\\.\" . | sort -u | head -15 | cut -c1-400" } Interpretability like this becoming increasingly important for enterprise agent deployments. s/o to @dan_biderman @realJessyLin & team for innovating on multiple dimensions here.

    1 Source

    @nikogrupenUnderrated interpretability result from our work with @EngramLab on @harvey law firm knowledge that makes a difference for the UX of long-horizon agents. The reasoning traces you get from parametric memory + study notes are much more intuitive than the interleaved bash commands you get from coding agents. When you read them (esp as a non-technical user), you can actually understand what’s going on. For example: "I remember two antitrust matters from earlier work, one that was terminated (client matter 1001-00004) and one that cleared after an HSR Second Request (client matter 1003-00003)" Compare this to the following from a coding agent (1 of 19 such bash commands btw): { "command": "cd /tmp/cache/1003-00003 && echo \"=== engagement letter head ===\" && sed -n '1,25p' Engagement/engagement-letter-hpe-fund-iv.docx.txt | cut -c1-500 && echo && echo \"=== practice group / partner mentions ===\" && grep -rhoi \"[^.]*\\(practice group\\|M&A partner\\|antitrust partner\\|responsible partner\\|lead partner\\)[^.]*\\.\" . | sort -u | head -12 | cut -c1-300 && echo && echo \"=== HSR outcome ===\" && grep -rhoi \"[^.]*\\(waiting period expired\\|expiration of the \\(extended \\)\\?waiting period\\|early termination\\|closed the investigation\\|closing letter\\|consent decree\\|no further action\\|without taking\\)[^.]*\\.\" . | sort -u | head -15 | cut -c1-400" } Interpretability like this becoming increasingly important for enterprise agent deployments. s/o to @dan_biderman @realJessyLin & team for innovating on multiple dimensions here.