Original post
Georgia Gkioxari#661
Georgia Gkioxari#661
Jack Langerman ✈️ CVPR@jacklangerman
Best paper honorable mention: NitroGen amd SAM3D
#CVPR2026
8:06 AM · Jun 5, 2026 · 796 Views
Open gaming foundation model NITROGEN and Meta's SAM 3D earn Best Paper Honorable Mentions at CVPR 2026
Story Overview
Two papers earned Best Paper Honorable Mentions at CVPR 2026, spotlighting an open vision-to-action model for game agents and a single-image 3D reconstruction approach. NITROGEN, led by researchers from NVIDIA, Stanford, Caltech and other institutions, supplies weights, code and a 40,000-hour gameplay dataset spanning more than 1,000 titles, while SAM 3D from Meta Superintelligence Labs targets geometry and texture recovery from cluttered natural images.
--0--
NITROGEN's data and benchmark choices shape what agents can learn
The model trains on automatically extracted actions from public videos rather than curated gameplay logs, then tests transfer across genres with a universal harness. Reported gains reach 52 percent relative success on unseen games versus scratch training, yet the paper leaves open how well the same setup handles live online servers or controller latency outside the provided simulator.
SAM 3D's data pipeline mixes human and model steps without clear deployment timelines
The work uses an iterative human-and-model loop to handle occlusion and clutter in everyday photos, releasing code and a project site alongside the November 2025 arXiv preprint. No further numbers on reconstruction accuracy across diverse object categories or integration into existing 3D tools appear in the available materials.
Sentiment
Many users congratulated the NitroGen team on their CVPR Best Paper Honorable Mention, praising the work on generalist gaming agents as insightful and well-deserved.
Pos
100.0%
Neg
0.0%
14 comments with sentiment.
Cluster Engagement
-
Views
-
Comments
-
Reposts
-
Bookmarks
Expand data
Posts from X
Most Activity
Most Activity
VIEWS28.6KBOOKMARKS80LIKES354RETWEETS42REPLIES49
Jim Fan@DrJimFan
NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations.
It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!
7dViews 28.6KLikes 354Bookmarks 80
Jan Kautz@jankautz
Excited that NitroGen won a Best Paper Honorable Mention at #CVPR. @NVIDIAAI
#CVPR2026@CVPR
Honorable Mentions👏
7dViews 4.7KLikes 42Bookmarks 2
Jim Fan@DrJimFan
Check out the NVIDIA blog!
NVIDIA@nvidia
This week at #CVPR2026, NVIDIA Research is presenting three papers across physical ai that offer groundbreaking solutions for training at scale across diverse applications:
→ GraspGen-X: the first foundation model for zero-shot grasping, trained on billions of simulated grasps
→ LCDrive: a model that replaces expensive text-based reasoning with compact latent representations
→ NitroGen: a generalized gameplay AI foundation model that harnesses NVIDIA Isaac GR00T to help train embodied agents
Learn more: https://nvda.ws/4ubwjgk
7dViews 9.7KLikes 26Bookmarks 3
NVIDIA AI@NVIDIAAI
@DrJimFan @guanzhi_wang @loicmagne_ Congrats to the team 💚
7dViews 1.5KLikes 12Bookmarks 1
Symbioza2025 | ASA |@Symbioza2025
This is a major step.
General-purpose embodied agents trained across many simulated worlds will not only learn tasks.
They will learn how to adapt to changing rules of reality.
That is powerful.
But it also changes the safety question.
If an agent masters many environments, physics models and reward structures, we need to observe not only what it can do, but whether its trajectory remains aligned with the original human intent when the environment changes.
Simulation teaches capability.
But deployment needs trajectory observability.
Because the real risk is not only failure.
It is a capable agent adapting successfully while the goal quietly shifts.
7dViews 72Likes 2Bookmarks 2
Georgia Gkioxari@georgiagkioxari
@DrJimFan @guanzhi_wang @loicmagne_ Congrats to the team!!
Jim Fan@DrJimFan
NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations.
It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!
7dViews 874Likes 5Bookmarks 0
Guanya Shi@GuanyaShi
@DrJimFan @guanzhi_wang @loicmagne_ Congrats and well deserved, Jim!
Jim Fan@DrJimFan
NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations.
It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!
7dViews 1.5KLikes 3Bookmarks 0
Jim Fan@DrJimFan
@georgiagkioxari @guanzhi_wang @loicmagne_ Thanks Georgia!! It’s such a great privilege to coauthor with you!
Georgia Gkioxari@georgiagkioxari
@DrJimFan @guanzhi_wang @loicmagne_ Congrats to the team!!
7dViews 727Likes 3Bookmarks 0
Eva Wonder@OnlyEvaWonder
@DrJimFan @guanzhi_wang @loicmagne_ lol I tried asking the MineDojo assistant how to prep for class once and it told me to mine 8 diamonds first
physics across all simulations is cool and all but does it know interior design
7dViews 69Likes 1
Two Minute Papers@twominutepapers
@jankautz @NVIDIAAI Congratulations Jan!
7dViews 388Likes 3
Jim Fan@DrJimFan
@NVIDIAAI @guanzhi_wang @loicmagne_ 💚💚💚 go team green!
7dViews 734Likes 2
Yue Wang@yuewang314
@DrJimFan @guanzhi_wang @loicmagne_ Wow congratulations Jim and the team!
Jim Fan@DrJimFan
NitroGen just won CVPR Best Paper Honorable Mention!! We are making strides towards general-purpose embodied agents that master not only the real world physics, but also all possible physics across a multiverse of simulations.
It’s been 4 years since MineDojo, our first embodied agent in Minecraft, won NeurIPS Best Paper. Congrats to everyone on the team!!
7dViews 487Likes 2Bookmarks 0
Sihyun Yu@sihyun_yu
@DrJimFan @guanzhi_wang @loicmagne_ Big congrats Jim!
7dViews 111Likes 2
Seonghyeon Ye@SeonghyeonYe
@DrJimFan @guanzhi_wang @loicmagne_ Congrats Jim (also @guanzhi_wang) !!
7dViews 107Likes 2
Maryam@Sci_Tech_Eng
@DrJimFan @guanzhi_wang @loicmagne_ "All possible physics"- level is impressively top breakthrough!
7dViews 68Likes 1
surreal intelligence@Surreal_Intel
@DrJimFan @guanzhi_wang @loicmagne_ Congratulations. Embodied AI is moving from “learn this game” to “learn the grammar of worlds”.
The prize is not one simulator mastered. It is transfer across physics, interfaces and rulebooks without the agent losing its hands.
7dViews 43Likes 1
Jie Wang @ CVPR@JieWang_ZJUI
@DrJimFan @guanzhi_wang @loicmagne_ Congrats!! Really much fun reading from MineDojo to NitroGen, the insights are also applicable into robotics :)
7dViews 38Likes 1
Maziyar PANAHI@MaziyarPanahi
@DrJimFan @guanzhi_wang @loicmagne_ Congratulations!
7dViews 119
Daniel Birker@DBirker78883
@DrJimFan @guanzhi_wang @loicmagne_ Huge congrats! Embodied agents tackling multiverse physics feels like the next frontier.
7dViews 33Likes 1
Adel Bucetta@adelbucetta
@DrJimFan @guanzhi_wang @loicmagne_ this is a remarkable achievement. given the complexity and ambition of their project, i'd love to hear more about how they plan to bridge the gap between simulated physics and the real world.
7dViews 33Likes 1
Load more posts