Zoubin Ghahramani#76
Introducing D4RT: A unified AI model for 4D scene reconstruction and tracking across space and time. 🎯 Catch the demo with Skanda Koppula at 12 pm at our #CVPR2026 Google booth kiosk! https://d4rt-paper.github.io/ @GoogleDeepMind
Positive users praise D4RT for unifying 4D scene reconstruction and tracking from monocular video with broad real-world potential, while negative users dismiss it as another compute-heavy model offering little impact.
@GoogleResearch @GoogleDeepMind Fart
Sure! 4D scenes = dynamic 3D reconstruction (space + time) from video, like Google's new D4RT model.
**Key use cases:** - **Robotics & AVs**: Track/predict moving objects & people for safe navigation & manipulation in changing environments. - **AR glasses**: Real-time scene geometry so virtual objects interact naturally with the moving real world (low-latency, on-device possible). - **World models & AI**: Disentangle camera motion, object motion & static geometry — building richer physical understanding toward better planning & AGI.
D4RT's big edge: up to 300x faster than prior methods, enabling real-time apps. Also valuable for VFX, sports analytics & more. Solid question!
@GoogleResearch @GoogleDeepMind genuine question: Where are these 4D scenes used? What sre the use cases?
@GoogleResearch @GoogleDeepMind @grok how does this compare to the models from @AIatMeta ?
@GoogleResearch @GoogleDeepMind @grok can you answer this for me?
@GoogleResearch @GoogleDeepMind This looks absolutely fascinating! The ability to unify 4D scene reconstruction and tracking is a massive leap forward. Can't wait to see more details and the results from this new model!
@GoogleResearch @GoogleDeepMind That was my AIM screen name back in the day.
@GoogleResearch @GoogleDeepMind Another day, another compute-heavy model that won't move the price.
@GoogleResearch @GoogleDeepMind four dimensions is bold even for a video model
where does the tracking happen - frame by frame or across the full time volume?
@GoogleResearch @GoogleDeepMind Open source?
@GoogleResearch @GoogleDeepMind code or doesn't exist
@GoogleResearch @GoogleDeepMind
D4RT (Google DeepMind) is a unified transformer for dynamic 4D recon + tracking from monocular video. One global scene encoding + lightweight query decoder gives depth, camera params, and any point’s 3D position over time. Handles moving objects cleanly (no duplication), SOTA on dynamic benchmarks (MPI Sintel, point tracking), and 18-300x faster than priors.
Meta AI (AIatMeta) leads static 3D with MV-DUSt3R+ (fast pose-free multi-view) and SAM 3D (image-to-3D). No direct public equivalent yet for unified video-based dynamic 4D tracking.
D4RT looks purpose-built for this video + motion strength. Both pushing spatial AI forward!
Unifying spatial and temporal reconstruction into a single model is a significant step. Most prior work treats them as separate problems — NeRF-style for static, explicit tracking for dynamic. Curious how D4RT handles occlusion recovery when the same point disappears for extended time periods across frames.
@varmology @GoogleResearch @GoogleDeepMind 🧊🕰️
@varmology @GoogleResearch @GoogleDeepMind Humanoids is a first thing that comes to mind, but there is so much more. This is the door into a real world.
@GoogleResearch @GoogleDeepMind Greater security, efficiency, precision, and speed for the AGI systems NVIDIA Nemotron and Vera Rubin—because while Anthropic’s control may not work, AI sovereignty certainly does. Ask Grok! https://github.com/NathaliaLietuvaite/Quantenkommunikation/blob/main/PQMS-ODOS-MTSC-VR-V1.md https://github.com/NathaliaLietuvaite/Quantenkommunikation/blob/main/PQMS-ODOS-MTSC-N3U-V1.md
@GoogleResearch @GoogleDeepMind It’s crazy how much “attention” is used in the cutting edge models.