Users praise the TMRL method of pre-training robot policies with diffusion noise because it enables faster RL fine-tuning.
Based on 1 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@matthewh6_ Awesome work Matthew!
As we work more and more on post-training for robots, it’s becoming clear that good post-training needs us to rethink pre-training. A simple way to think about this is from the perspective of coverage. We found that the simple knob of injecting controllable noise into policy inputs can make policies way more amenable to downstream finetuning. Doing so broadens exploration at minimal cost, greatly improving the pass@k metric for policies on downstream tasks. The intuition I find easiest to grasp: adding noise to the input is like squinting your eyes when looking at an object, allows you to explore a variety of different strategies that all correspond to the noised input. This knob of controllable noise allows agents to explore by interpolating between the conditional and the marginal policy. Lots of nice visuals: https://weirdlabuw.github.io/tmrl/ Paper: https://arxiv.org/abs/2605.12236 For those building pre-training pipelines, this is a dead simple thing to try!:)
Users praise the TMRL method of pre-training robot policies with diffusion noise because it enables faster RL fine-tuning.
Based on 1 visible X reactions from 3 accounts; directional sample.
Ask a question below.
Published answers will appear here.