New PQSG Framework Evaluates Physics Plausibility In Text-To-Video AI

Method Here's a real example. Prompt: "a brown tennis ball suspended by a grabber tool, dropped onto a pillow." The grabber lets go, the ball plops down, bounces once, and the pillow doesn't move. No dent, no compression. A human knows that's wrong. The question is how a machine knows. 👇

Huge shoutout to @jmin__cho @zhan1624 @EliasEskin @mohitban47 for their phenomenal support and mentorship throughout.

Turns out AI video can render a perfect pillow, then forget it should dent when a ball lands on it. These physics slip-ups are everywhere once you start looking, and they're exactly what current metrics tend to miss.

We built PQSG to catch exactly these physics failures, and I'm thrilled to see it accepted to #ECCV2026! Thread👇

Dataset Many benchmarks hand you one score per video. FinePhyEval gives you four: object, action, physics, and overall. 195 videos from Sora 2, Veo 3, and Wan 2.1, every one scored by humans. That separation is what lets us measure physics on its own.

Look closely at modern AI video generators, and you'll notice subtle (and not-so-subtle) physics slip-ups that standard metrics completely miss.

To pinpoint these errors, we introduce our #ECCV2026 work, Physics Question Scene Graph (PQSG), a framework that evaluates AI-generated videos for physical realism, as well as objects and actions, using fine-grained evaluation questions and dependencies.

We also release FinePhyEval, a fine-grained evaluation benchmark. It has evaluated leading models like Sora, Veo, Wan, and Cosmos-Predict, and now it's time to evaluate yours!

Great work led by @atinpothiraj. Check more 🧵👇

If we want video generation to act as a world model, models must follow physics laws – something current models struggle with. Many metrics miss these failures, so in our new #ECCV2026 paper we introduced PQSG:

PQSG measures physical realism by decomposing a prompt into a graph of object, action, and physics questions and their dependencies, leading to fine-grained eval of exactly what went wrong in a video! Scores correlate better with human judgments and will continue to improve w/ better VLMs!

🧵👇

Framework The trick: don't ask "is this realistic?" Ask 20+ yes/no questions in a tree. Is the ball suspended from the grabber? (Yes) ↓ Does the grabber release it? (Yes) ↓ Does the pillow deform on impact? The pillow never deforms, so that's a NO. If the grabber never released it, we don’t even ask the question about the pillow deforming. One failure up top knocks out the whole branch. No hallucinated answers about a fall that never happened.

Bonus Best part: PQSG doesn't just grade videos. It helps us to improve them. Feed its feedback back into the prompt and quality jumps ~15% in a single pass. No retraining.

Results What we found: -- Video models are great at putting objects in the frame. They fall apart on action and physics.

-- PQSG is better aligned with human judgment than prior metrics, and physics is the single thing humans weigh most when they call a video real or fake.

-- Sora 2 and Veo 3 pull well ahead of the open-source models.

Awesome @eccvconf project led by @atinpothiraj and in collaboration with @jmin__cho @EliasEskin @mohitban47

@unc_ai_group @unccs @allen_ai @JHUCompSci @UTCompSci

- Paper: https://arxiv.org/abs/2606.25306 - HF Paper: https://huggingface.co/papers/2606.25306 - Code / Data: https://github.com/atinpothiraj/pqsg

@AtinPothiraj That's my GOAT! 🐐

@AtinPothiraj Congrats Atin! Great work

@AtinPothiraj 🐐

@AtinPothiraj Love this

@AtinPothiraj This is sick

@AtinPothiraj Awesome stuff

@AtinPothiraj Interesting work Atin!

@jmin__cho In course video that error is not a visual glitch — it is a broken learning path. A wrong formula in episode 14 means a student fails the exam. Deterministic rendering from source documents sidesteps the problem: the formula is correct because code renders it, not a prompt.

@AtinPothiraj Great work Atin!