• Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    AI
    Announcement

    Math Verify Score calculation reportedly fixed in LM Evaluation Harness

    A researcher says they found and fixed the calculation and suggests rerunning scores reported before roughly January 2026.

    Rylan SchaefferRS
    1 Source, 1h ago, first seen 1h ago

    TLDR

    A researcher says they and collaborators found and fixed how Math Verify Score was computed in the LM Evaluation Harness. They suggest papers reporting scores before roughly January 2026 should probably rerun their numbers. They also linked to an arXiv paper and credited collaborators and advisors for the work.

    Combined views

    54

    1 Source, first seen 1h ago

    1 likes

    Combined views

    54

    1 Source, first seen 1h ago

    1 likes

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    Rylan Schaeffer@RylanSchaefferWe also found and fixed how Math Verify Score was computed in the LM Evaluation Harness. Any paper reporting scores before ~Jan 2026 should probably have their numbers rerun 8/81h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    1 Source

    Rylan Schaeffer@RylanSchaefferWe also found and fixed how Math Verify Score was computed in the LM Evaluation Harness. Any paper reporting scores before ~Jan 2026 should probably have their numbers rerun 8/81h
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet