• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Odyssey Launches CaliBench for World Model Randomness

    Benchmark evaluates whether video world models reproduce real physical outcome distributions.

    AK
    OC
    SA
    6 Sources, 42d ago, first seen 42d ago

    TLDR

    Odyssey posted an announcement introducing CaliBench. The benchmark evaluates whether video world models reproduce the true randomness of physical events such as dice rolls or card draws. A linked page on the company site describes the test as checking if models produce outcome distributions that match reality rather than only appearing physically plausible frame by frame. Visible replies include one noting a generated video where an apple floated instead of falling and another observing that models show biases toward certain outcomes.

    Combined views

    128.2K

    6 Sources, first seen 42d ago

    Combined views

    128.2K

    6 Sources, first seen 42d ago

    179 likes
    179 likes
    11 comments
    61 saves
    39 reposts
    Oliver Cameron

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source
    11 comments
    61 saves
    39 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    6 Sources

    @odysseymlToday, we're introducing CaliBench, evaluating whether video world models reproduce the true randomness of our universe. Roll a dice or pick a card, the outcomes produced by a world model should match reality. Find out if do! https://odyssey.ml/introducing-calibench
    @_akhaliqRT @odysseyml: Today, we're introducing CaliBench, evaluating whether video world models reproduce the true randomness of our universe. Ro…
    @olivercameron@svpino @odysseyml Correct. Biases to certain outcomes all over the place.
    @svpino@olivercameron I remember generating a video with Seadance where gravity didn't exist. I had a pirate that dropped an apple and the thing just stayed floating :)

    6 Sources

    @odysseymlToday, we're introducing CaliBench, evaluating whether video world models reproduce the true randomness of our universe. Roll a dice or pick a card, the outcomes produced by a world model should match reality. Find out if do! https://odyssey.ml/introducing-calibench
    @_akhaliqRT @odysseyml: Today, we're introducing CaliBench, evaluating whether video world models reproduce the true randomness of our universe. Ro…
    @olivercameron@svpino @odysseyml Correct. Biases to certain outcomes all over the place.
    @svpino@olivercameron I remember generating a video with Seadance where gravity didn't exist. I had a pirate that dropped an apple and the thing just stayed floating :)

    Related

    World Models Create Infinite AI Learning Environments

    Odyssey ML CEO Oliver Cameron describes world models as dynamic environments for training AI agents.