NVIDIA open-sources Cosmos 3, an 8B and 32B physical AI omnimodel combining vision reasoning, world simulation, and action generation

VIEWS1MBOOKMARKS396LIKES873REPLIES50

This is THE moment of Physical AI!

We are officially announcing Cosmos 3: Omnimodal World Models for Physical AI 🚀

- Cosmos 3 is an omnimodal world model: within a unified architecture, it can understand and generate language, images, video, audio, and actions. - It is not just a VLM, not just a video generator, not just an audio-visual generative model, and not just a physics simulator / world-action model. It can understand images and videos, generate images, videos, and audio, simulate future worlds, predict actions, and generate robot policies—enabling models to truly begin to “touch the world.” - Cosmos 3 is the #1 open-weight reasoner / T2I / I2V / robot policy across many benchmarks.

Huge thanks to every teammate who fought side by side on this journey—from architecture, data, training, infra, serving, and evaluation to post-training. Every part of this project carries an incredible amount of hard work. This was my first time leading a project as Tech Lead, and I feel truly fortunate.

The future of Physical AI needs models that can not only “see” and “describe” the world, but also “imagine,” “simulate,” and “act”—and eventually close the loop with the real world. I hope Cosmos 3 can become an important starting point for this direction, and I’m excited to push Physical AI into its next stage together with the open-source community.

Welcome to the era of Physical AI.

HuggingFace: https://huggingface.co/collections/nvidia/cosmos3 Project Website: https://research.nvidia.com/labs/cosmos-lab/cosmos3 Code: https://github.com/NVIDIA/cosmos

29d1M873396

RETWEETS308

NVIDIA AI@NVIDIAAI

Introducing Cosmos 3: Our latest frontier model for Physical AI

Cosmos 3 is the world’s first fully open omnimodel with native vision reasoning, world and action generation.

Today we’re releasing Super (32B) and Nano (8B) variants.

29d319.4K2.6K1K

Artificial Analysis@ArtificialAnlys

NVIDIA's Cosmos 3 lands at #1 among open weights models in both Text to Image and Image to Video on the Artificial Analysis Leaderboards!

Cosmos 3 is a family of omnimodal world models for Physical AI from @nvidia, unifying language, image, video, audio and action in a single Mixture-of-Transformers architecture that pairs an autoregressive reasoner with a diffusion generator.

The family comes in four variants: base Nano (16B: 8B reasoner tower + 8B generator tower) and Super (64B: 32B reasoner tower + 32B generator tower) models, with the Super model also having Text2Image and Image2Video fine-tuned variants, which are the versions listed in the Artificial Analysis Arena Leaderboards.

Cosmos3-Super-Text2Image (agentic) runs through an agentic prompt-upsampling harness, and takes the #1 open weights spot in Text to Image, surpassing HiDream-O1-Image-Dev-2604, Alibaba's Qwen Image Max 2512 and Black Forest Labs' FLUX.2 [dev].

Cosmos3-Super-Image2Video takes #1 open weights in Image to Video (No Audio), ahead of Lightricks' LTX-2, and Alibaba's Wan 2.2 A14B.

Cosmos 3 generators take structured JSON prompts rather than plain text, so prompt upsampling is needed to reproduce these results. This upsampling can be handled by an external harness or by the model's own reasoner branch, so it can also run self-contained.

Cosmos 3 is fully open under the OpenMDW 1.1 license, shipping with weights, code, curated datasets and fine-tuning recipes available on @huggingface. First-party and third-party APIs are expected over the next few weeks, with pricing to follow.

See the thread below for example generations and a link to try Cosmos 3 in our arena 🧵

28d32.2K30784

Chubby♨️@kimmonismus

1/ NVIDIA just open-sourced Cosmos 3 at GTC Taipei!

It's the first fully open "omnimodel" for physical AI - one model that understands the real world, predicts what happens next, and generates the actions a robot should take.

Weights, code, datasets. All open. And this is really big. Lets dig into everything: 🧵

NVIDIA AI@NVIDIAAI

Introducing Cosmos 3: Our latest frontier model for Physical AI

Cosmos 3 is the world’s first fully open omnimodel with native vision reasoning, world and action generation.

Today we’re releasing Super (32B) and Nano (8B) variants.

28d16.9K16346

Ashkan Mirzaei@ashmrz10

I’m excited to share what our team has been building at @NVIDIAAI since I joined: Cosmos 3, an omnimodal world model for Physical AI.

Project: https://research.nvidia.com/labs/cosmos-lab/cosmos3 HF: https://huggingface.co/collections/nvidia/cosmos3 Code: https://github.com/NVIDIA/cosmos

28d10.8K14844

Turing Post@TheTuringPost

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯

Why is Physical AI much harder than building a chatbot?

Understanding the world is not enough, robots need to predict it and act inside it.

That's the idea behind NVIDIA Cosmos 3:

→ Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal.

Previous systems often stitched these capabilities together using separate models.

Cosmos 3 combines them into a single architecture with two components:

▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context.

▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints.

So Cosmos 3 moves from:

Perception → Model A Prediction → Model B Actions → Model C

to:

Perception + Prediction + Actions → One unified system.

The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions:

1. What is happening? 2. What will happen next? 3. What should I do?

That's a big shift from today's AI, which mostly focuses on generating text.

And check the benchmarks!

Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

29d11.9K7444

merve@mervenoyann

NVIDIA just dropped Cosmos 3 at GTC 🔥

closest thing to AGI as world model > it can reason, understand AND generate videos, images, actions, text > sota, comes in 16B, 65B, with datasets > diffusers support 🧨 > open license 🤗

NVIDIA AI@NVIDIAAI

Introducing Cosmos 3: Our latest frontier model for Physical AI

Cosmos 3 is the world’s first fully open omnimodel with native vision reasoning, world and action generation.

Today we’re releasing Super (32B) and Nano (8B) variants.

29d7.5K7130

NVIDIA AI@NVIDIAAI

Cosmos 3 ties everything together.

Previous releases separated world generation, physical understanding, and controlled scene generation. Cosmos 3’s MoT architecture unifies these capabilities by pairing an autoregressive reasoner tower with a diffusion-based generator tower.

NVIDIA AI@NVIDIAAI

Introducing Cosmos 3: Our latest frontier model for Physical AI

Cosmos 3 is the world’s first fully open omnimodel with native vision reasoning, world and action generation.

Today we’re releasing Super (32B) and Nano (8B) variants.

29d9.3K12418

Zekun Hao@zekun_hao

Look what we’re cooking! Cosmos 3 is a family of unified omnimodal world model (language, image, video, audio, action), topping multiple benchmarks! Proud to have led Cosmos3-Super-Image2Video, now the #1 open I2V model on Artificial Analysis. Hope it empowers the community!

29d4.2K609

Turing Post@TheTuringPost

NVIDIA's Cosmos 3 is what we've never seen before ‒ an Omnimodal World Model. It's closing the loop for physical AI.

All stack in one system: world and multimodal understanding, future generation, reasoning and action

This is the next step in Jensen Huang’s AI progression: perception AI → generative AI → agentic AI → physical AI

Here is what you need to know about it ↓

Turing Post@TheTuringPost

Breaking news: Cosmos 3 is here. They are attempting to do something completely new 🤯

Why is Physical AI much harder than building a chatbot?

Understanding the world is not enough, robots need to predict it and act inside it.

That's the idea behind NVIDIA Cosmos 3:

→ Reasoning model understands what's happening from video, images, text, and actions. → World model generates future states of the environment. → Action model generates the actions needed to achieve a goal.

Previous systems often stitched these capabilities together using separate models.

Cosmos 3 combines them into a single architecture with two components:

▪️ Reasoner Tower Analyzes observations and builds an understanding of objects, motion, interactions, and physical context.

▪️ Generator Tower Uses that understanding to generate future videos and action sequences that obey physical constraints.

So Cosmos 3 moves from:

Perception → Model A Prediction → Model B Actions → Model C

to:

Perception + Prediction + Actions → One unified system.

The goal is to make robots, autonomous vehicles, and smart environments better at answering three questions:

1. What is happening? 2. What will happen next? 3. What should I do?

That's a big shift from today's AI, which mostly focuses on generating text.

And check the benchmarks!

Physical AI needs to generate decisions that survive contact with the real world. 🚗🤖

28d3.8K2810

Victoria X Lin@VictoriaLinML

Excited to see 🌟 Mixture-of-Transformers (MoT) ideas continue to scale in native multimodal systems. https://www.nvidia.com/en-gb/glossary/mixture-of-transformers/

In our MoT work, we explored modality-specialized transformer parameters as a path toward building more efficient multimodal foundation models with stronger scaling properties. Cosmos 3 is an impressive instantiation of this design space in physical AI — combining reasoning, world generation, and action within a unified architecture.

Congrats to @NVIDIAAI and the broader team on this exciting open-source release!

w/ @liang_weixin @liliyu_lili Liang Luo and the team

NVIDIA AI@NVIDIAAI

Cosmos 3 ties everything together.

Previous releases separated world generation, physical understanding, and controlled scene generation. Cosmos 3’s MoT architecture unifies these capabilities by pairing an autoregressive reasoner tower with a diffusion-based generator tower.

27d3.3K3115

Ashkan Mirzaei@ashmrz10

Super proud of the team behind Cosmos 3 at @NVIDIAAI 🚀💚

Frontier-level text-to-image performance, fully open source, and available for anyone to try today 🌍✨

I’m incredibly grateful to have been part of this effort 🙏 and can’t wait to see what people build with it.

28d4.4K542

Ming-Yu Liu@liu_mingyu

Cosmos 3 is a family of omnimodal world models designed to jointly process and generate language, image, video, audio, and action sequences within a unified mixture-of-transformers architecture. It has incredible capabilities and is ranked as the number one open-source Text2Image and Image2Video model by Artificial Analysis, and as the number one robot policy model by RoboLab and RoboArena. Try it out.

model: https://huggingface.co/collections/nvidia/cosmos3 code: https://github.com/NVIDIA/cosmos website: https://research.nvidia.com/labs/cosmos-lab/cosmos3/ paper: https://research.nvidia.com/labs/cosmos-lab/cosmos3/technical-report.pdf

NVIDIA AI@NVIDIAAI

Introducing Cosmos 3: Our latest frontier model for Physical AI

Cosmos 3 is the world’s first fully open omnimodel with native vision reasoning, world and action generation.

Today we’re releasing Super (32B) and Nano (8B) variants.

29d22K19553

Turing Post@TheTuringPost

Very interesting Mixture-of-Transformers (MoT) architecture of Cosmos 3

29d1.3K258

Hao Zhang@haozhangml

Really really impressive model; thanks for opening source it!!

Ming-Yu Liu@liu_mingyu

Introducing NVIDIA Cosmos 3

We released NVIDIA Cosmos 3 last night.

And today, seeing it take the top spots across 8+ open model leaderboards feels surreal. We spent months working towards this moment.

Here’s the breakdown:

The Leaderboard Wins

World Reasoning 🏆 #1 open model on VANTAGE-Bench for vision AI 🏆 #1 overall on Traffic Anomaly Reasoning (TAR)

World Generation 🏆 #1 open model on Artificial Analysis Image-to-Video leaderboard 🏆 #1 open model on Artificial Analysis Text-to-Image leaderboard 🏆 #1 open model on PAI-Bench for physical AI synthetic data generation 🏆 #1 open model on Physics-IQ, which measures accuracy on physical laws 🏆 #1 open model on R-Bench for world generation quality

World Action 🏆 #1 on RoboArena for specialized policy 🏆 #1 on RoboLab for action generation

But the leaderboards are only part of the story. The real story is why we built Cosmos 3 in the first place.

The Problem

Training robots and autonomous systems in the real world is painfully hard.

Robots need to try the same thing numerous times before they succeed reliably. Self-driving cars need rare edge cases that may never happen naturally. Smart machines need to understand physics, motion, contact, failure, and surprise.

And real-world data is slow, expensive, and sometimes dangerous to collect. At some point, the answer cannot just be “collect more data.”

You can’t collect your way out of an infinite physical world. You have to generate it.

That… was the question behind Cosmos: Can one model understand the physical world deeply enough to reason about it, simulate it, and generate actions inside it?

What We Built

Cosmos 3 is the first omni-model for physical AI. It can understand and generate across: language · images · video · audio · action sequences

It is not just a VLM.

Not just a video generator.

Not just a robot policy model.

It is all of them, in one single model.

That matters because physical AI has been fragmented for a long time. Cosmos 3 is our attempt to collapse that fragmentation.

Depending on how you configure the inputs and outputs, the same model can act as a vision-language model, a video/world generator, a world simulator, or a world-action model.

No separate architecture required.

The Architecture

Under the hood, Cosmos 3 uses a dual-tower Mixture-of-Transformers architecture.

One tower is autoregressive for reasoning. It handles next-token prediction for language and discrete understanding.

The other tower is diffusion-based- for generation. It denoises images, video, audio, and action trajectories. Two towers. Dual-stream joint attention. One shared world representation.

Each modality gets its own tools: visual encoders, video VAEs, audio VAEs, and action projectors that can map different embodiments into a unified action space.

Action is a first-class modality in Cosmos 3.

That’s what makes it more than a video model. It doesn’t just predict and generate what the world might look like. It can connect reasoning and world modeling to physically grounded action.

Why This Matters

One of the most interesting findings from the ablation work is that training action domains together creates positive transfer.

That means adding more embodiments does not just add more use cases. It can actually make the model better.

This is the heart of why omnimodal training matters.

A shared world representation is not just convenient. It can make each individual task stronger. That’s the part that feels like the beginning of something much bigger.

The part I’m most excited about is that Cosmos 3 is fully open.

Developers get the models, scripts, optimization, inference endpoints, post-training recipes, datasets, and benchmarks.

Everything is available under the Linux Foundation’s OpenMDW 1.1 License.

You can use Cosmos 3 out of the box. You can use the VLM, world model, or world-action pieces separately.

You can post-train it for your own domain, embodiment, or accuracy target.

That’s what makes this feel different.

Cosmos 3 is not just a model release. It is the foundation for building intelligence for autonomous machines.

For me, Cosmos 3 feels like a step toward a world where physical AI development becomes much more scalable and accessible - to a new age of developers and agents.

That’s what we built Cosmos 3 for. I cannot wait to see what you build with it.

Download Models on Hugging Face https://huggingface.co/collections/nvidia/cosmos3

Customize Models on GitHub https://github.com/NVIDIA/cosmos

Read the Tech Blog to Learn More https://developer.nvidia.com/blog/develop-physical-ai-reasoning-world-and-action-models-with-nvidia-cosmos-3

28d4.1K317

NVIDIA AI@NVIDIAAI

@ArtificialAnlys Image-to-video generation is just as impressive.

Input image: "Generate a 16:9 image from a dashcam view of a formula 1 racing event"

Video prompt: "A high-speed racing event where a car navigates multiple winding turns"

🔊 Sound on - generated by Cosmos 3.

29d1.8K156

NVIDIA AI@NVIDIAAI

@ArtificialAnlys As always, Cosmos 3 is fully open. This includes model weights and post-training recipes.

Available now on @huggingface https://huggingface.co/collections/nvidia/cosmos3

29d1.7K206

Chubby♨️@kimmonismus

2/ What changed: old Cosmos split the work across separate models — one to understand a scene, one to generate video, one for controlled simulation. Cosmos 3 fuses everything into a single Mixture-of-Transformers with two towers: → a reasoner (the VLM "brain") → a diffusion generator (the "imagination")

Chubby♨️@kimmonismus

1/ NVIDIA just open-sourced Cosmos 3 at GTC Taipei!

It's the first fully open "omnimodel" for physical AI - one model that understands the real world, predicts what happens next, and generates the actions a robot should take.

Weights, code, datasets. All open. And this is really big. Lets dig into everything: 🧵

28d2.9K143

NVIDIA AI@NVIDIAAI

Delivering leading results on physical AI benchmarks among open models, it ranks first across @ArtificialAnlys, Physics-IQ, PAI-Bench and R-Bench for world generation accuracy, RoboLab and RoboArena for action policy and the VANTAGE-Bench and TAR leaderboards for vision understanding.

29d1K172

Artificial Analysis@ArtificialAnlys

Cosmos3-Super-Text2Image (agentic) is the #1 open weights model in the Artificial Analysis Text to Image Leaderboard

28d1.9K162