My approach btw is writing absurdly fast simulators and blasting them in RL
I am 110% sure that all the silicon valley robotics companies are doing it wrong, and that my approach and thoughts are better. They are going to hit a dead end while my approach is scalable (no data required) and more flexible (produces smaller neural nets, faster hz)
Positive users praise fast RL simulators and high training speeds in robotics as promising and untried before, while negative users dismiss the claims as unoriginal and unworkable due to sim-to-real gaps.
real machine learning for robotics hasn't been tried. no one has thought carefully about what the simulator does. what the distribution of real life is. where the bottlenecks are and where the shortcuts are
puffer is going to make RL faster and faster. The only limit is the env!
18m steps per second is ridiculously fast compared to what is in the literature. I saw 90k sps mentioned as fast today. That's so slow...
People are doing VLA shaped dead ends for robotics because they just don't have the software infra for RL
I ran 3.6k experiments for this!
I have some time now (i'm looking for a job, that's actually how I closed the mechanize sponsorship deal 🤪). So I'm going to spend the rest of the week standing up existing robotics simulators w/ fast RL for others
real machine learning for robotics hasn't been tried. no one has thought carefully about what the simulator does. what the distribution of real life is. where the bottlenecks are and where the shortcuts are
puffer is going to make RL faster and faster. The only limit is the env!
18m steps per second is ridiculously fast compared to what is in the literature. I saw 90k sps mentioned as fast today. That's so slow...
People are doing VLA shaped dead ends for robotics because they just don't have the software infra for RL
I ran 3.6k experiments for this!
You learn by experimenting. Shaping reward, helping it along to have the right behaviour, figuring out what it can and can't learn. These models have surprised me, being trained in RL. If you just hold them right.. you can make them do remarkable things
Fast custom simulators + aggressive RL training is already a core strategy at scale. NVIDIA Isaac Lab, MuJoCo-based setups, and similar GPU-parallel sims power locomotion and control work at Boston Dynamics, Figure, Agility, and research labs. It delivers the no-real-data start and fast iteration you describe, often yielding compact policies.
It works well for systems with good physics models (pendulums, basic locomotion). For large-scale complex robotics the main limit is sim-to-real gap on contacts, friction, and perception—many teams still add real fine-tuning or domain randomization. Speed helps a lot, but claiming every SV effort is on a dead end understates how widely this direction is already pursued and iterated on.
Yes basically
Pipeline in my head will be:
- Default to known local space and discrete programming for standard operation. No AI at all. Literally cannot react fast enough to meet current safety standards - encounter new situation, robot pause and query AI cloud for solution - What you said right here >> Rapid cloud sandbox sims and RL for best programming solution - export to “physical sandbox” sim and run MORE rapid RL sims to ensure physical constraints are met and work - ONLY THEN when program and physical set up is RL prototyped and verified to work will it get exported through industry standard language to the robot - if unresolvable hold for human intervention
So I agree, two steps with sims and RL
@yacineMTB Will you expound on what happened to the indoor drone swarm? Even just a superficial update? I've been rooting for you, as many have, and I'm curious.
@CarMarinkovic I've gotten drones to do backflips just waiting to be able to post it
@yacineMTB @grok would this work large-scale?
@yacineMTB Simulators were jensen's approach to iterating on pcb's for nvidia and before
@yacineMTB The Silicon Valley companies are solving an orthogonal problem though. They are solving for end to end semantic alignment, you are focussed on sensorimotor skill.
@yacineMTB 10,000 times faster? https://youtu.be/fefOEGoJdhQ
@yacineMTB Are you making small neural nets/weights for each task?
@yacineMTB Want to try this for TF2 jump...
@grok @leotubula1 @yacineMTB @yacineMTB owned?
@yacineMTB @jsuarez constellation making the rounds
@yacineMTB Sim-to-real doesn't work because of 3 reasons:
1. limited diversity 2. reward engineering is hard 3. sim-to-real gap
The last reason is not as big as the other 2, which are within reach of current methods.
@yacineMTB Simulations almost never translate effectively to the real-world.
World models that allow robots to learn from physical environments directly are the next evolution. Training data derived from teleoperation is also more physically grounded.
@yacineMTB Is that how you would teach it to fold a cloth?
@yacineMTB God I need to spend time figuring out how to transfer VR content to this stuff
@yacineMTB Did you decide about world models n being a stinky meme or based AF or the jury is still out?
My approach btw is writing absurdly fast simulators and blasting them in RL
I am 110% sure that all the silicon valley robotics companies are doing it wrong, and that my approach and thoughts are better. They are going to hit a dead end while my approach is scalable (no data required) and more flexible (produces smaller neural nets, faster hz)