• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI

HyperFrames announces Code2Video Bench for AI video tasks

HyperFrames says it built the benchmark with Google DeepMind and Kaggle, drawing a distinction between coding for SWE-bench and generating video that “feels alive.”

Omar SansevieroOS
HyperFramesHY
8 Sources, 20d ago, first seen 20d ago

TLDR

HyperFrames announced Code2Video Bench, saying it built the benchmark in collaboration with Google DeepMind and Kaggle. It argues that SWE-bench coding and creating lively video through code are different challenges, and presents the benchmark as a way to help frontier AI labs improve at video tasks handled by agents.

Combined views

1.6M

8 Sources, first seen 20d ago

1.3K likes470 comments657 saves649 reposts

Combined views

1.6M

8 Sources, first seen 20d ago

1.3K likes470 comments657 saves649 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

8 Sources

HyperFrames@HyperFrames_if you tried hyperframes, you would know the agentic video stack is being built on code-gen but SWE-bench code v.s. code-to-video that feels alive are two different things we built the Code2Video Bench, in collab w/ Google DeepMind & @Kaggle frontier labs can finally get good at agentic video tasks20d
Omar Sanseviero@osansevieroRT @HyperFrames_: if you tried hyperframes, you would know the agentic video stack is being built on code-gen but SWE-bench code v.s. code…20d
Kaggle@kaggleHow well can frontier models actually code human-preferred motion graphics? 🎬⚡️ We’re excited to introduce @HeyGen's Code2Video Bench on Kaggle, a standardized benchmark measuring how well models can generate deterministic, high-quality @HyperFrames_ code from natural language creative briefs. 💡 The Problem: Static image benchmarks and traditional video generation evaluations don’t measure whether code generation models can follow detailed creative briefs to programmatically control timing, animation sequences, and asset placement over time. 🛠️ How it Works: 1. Creative Brief: Models receive commercial-grade motion design prompts (example: “Create a 6.8s Figma-style product hook that zooms out from a blooming vector flower into a 3-card web page and live timeline editor”). 2. Code to MP4: The model generates HTML/CSS/JS, deterministically rendered into output.mp4 via the open-source Hyperframes engine in isolated Kaggle Harbor sandboxes. 3. The Auto-Judge: HeyGen’s in-house model architecture, trained on human preferences across a 5-axis evaluation framework, evaluates the rendered MP4. Give it the original prompt and two candidate videos, and it returns the probability a human would prefer Video A over Video B. It does this once per axis — five independent verdicts, not one blended score. Here’s what we’ve learned so far 🔽20d
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    8 Sources

    HyperFrames@HyperFrames_if you tried hyperframes, you would know the agentic video stack is being built on code-gen but SWE-bench code v.s. code-to-video that feels alive are two different things we built the Code2Video Bench, in collab w/ Google DeepMind & @Kaggle frontier labs can finally get good at agentic video tasks20d
    Omar Sanseviero@osansevieroRT @HyperFrames_: if you tried hyperframes, you would know the agentic video stack is being built on code-gen but SWE-bench code v.s. code…20d
    Kaggle@kaggleHow well can frontier models actually code human-preferred motion graphics? 🎬⚡️ We’re excited to introduce @HeyGen's Code2Video Bench on Kaggle, a standardized benchmark measuring how well models can generate deterministic, high-quality @HyperFrames_ code from natural language creative briefs. 💡 The Problem: Static image benchmarks and traditional video generation evaluations don’t measure whether code generation models can follow detailed creative briefs to programmatically control timing, animation sequences, and asset placement over time. 🛠️ How it Works: 1. Creative Brief: Models receive commercial-grade motion design prompts (example: “Create a 6.8s Figma-style product hook that zooms out from a blooming vector flower into a 3-card web page and live timeline editor”). 2. Code to MP4: The model generates HTML/CSS/JS, deterministically rendered into output.mp4 via the open-source Hyperframes engine in isolated Kaggle Harbor sandboxes. 3. The Auto-Judge: HeyGen’s in-house model architecture, trained on human preferences across a 5-axis evaluation framework, evaluates the rendered MP4. Give it the original prompt and two candidate videos, and it returns the probability a human would prefer Video A over Video B. It does this once per axis — five independent verdicts, not one blended score. Here’s what we’ve learned so far 🔽20d
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet