Reactions from ranked influencers
11 posts
Join our team to work on Embodied World-Models with @MustafaShukor1 (co-author VL-JEPA with @ylecun ) Link to apply in comments 😇
We're selected among 2% of startups for SPRIND's Next Frontier AI Challenge This competitive program brings €125M to support teams building new AI paradigms.
Click Join us to list open positions https://uma.bot
The SPRIND grant will accelerate UMA’s effort to develop latent world models for humanoid robots. Throughout my career I’ve kept the objective of scaling up learning using minimal supervision and efficient data collection. This led me to propose self-supervision methods in latent space such as Time-Contrastive Networks, which demonstrated 3rd person imitation of a human entirely without labels: https://sermanet.github.io/imitate/ One key component was co-training different modalities, viewpoints and embodiments into a single latent space, entirely from raw unlabeled videos. The dimensions of the world discovered by the models ended up naturally aligning in latent space. This latent alignment across viewpoints and embodiments is what unlocked imitating humans directly from 3rd-person raw pixels, similarly to the concept of “mirror neurons” firing when observing someone else performing a task. We later developed self-supervised methods with @coreylynch for generating actions by pulling together in latent space the representations of (start, goal) image pairs and sequence of images [start, …, goal]. We combined this label-free learning method with an efficient data collection method: playing. Play is an efficient way to discover the world by leveraging existing knowledge and skills. Humans and animals use it to learn about the world and practice in advance. We called this Learning Latent Plans from Play: https://learning-from-play.github.io/ We subsequently augmented this approach with language representations so that you could control the latent space via language conditioning. But most of the learning was coming from self-supervising on video, language labels accounted for less than 1% of data: https://sermanet.github.io/language-play/ I still believe that learning the world in latent space by mostly self-supervising on raw unlabeled data is the way to go. We now have stronger unsupervised learning methods, more capable hardware and better data acquisition means, it is an exciting time to keep pushing that vision. If that vision speaks to you, we are hiring for our Embodied World Models team: Research Scientist: https://app.dover.com/apply/3a4a12c9-b2ec-406b-b5c0-8c64e9024687/e223cbc4-8b0e-4b3b-9935-47d046af65b3/ Research Engineer: https://app.dover.com/apply/3a4a12c9-b2ec-406b-b5c0-8c64e9024687/1bb6bdb5-dd1d-49d5-8496-45bebaa43f3d/
Join our team to work on Embodied World-Models with @MustafaShukor1 (co-author VL-JEPA with @ylecun ) Link to apply in comments 😇 https://twitter.com/UMA_Robots/status/2078061302572876143
We’re taking a different bet from most frontier robotics labs: scaling latent-space world models for physical AI. 🧵 1/n
We're selected among 2% of startups for SPRIND's Next Frontier AI Challenge This competitive program brings €125M to support teams building new AI paradigms.
Thanks ;) We opened positions on World Model to work with @MustafaShukor1 (co-author of VL-JEPA with Yann LeCun) https://x.com/UMA_Robots/status/2078061302572876143
Congrats to all teams! https://twitter.com/sprind/status/2078057133510127974
The heavy boost in world models track for our team 🤘🎉🔥
We're selected among 2% of startups for SPRIND's Next Frontier AI Challenge This competitive program brings €125M to support teams building new AI paradigms.