Microsoft AI Frontiers researchers introduce ECHO, a method that adds environment prediction loss to GRPO training for command-line agents to build internal world models
Tests on Qwen3 models showed higher benchmark pass rates.
@DimitrisPapail @ChengleiSi This is nice work, so sorry for distracting from the substance, but I'm genuinely curious:
> This work was done at AI Frontiers, a boutique research lab inside Microsoft Research.
What does "boutique research lab" mean here?
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@AlexGDimakis @DimitrisPapail @ChengleiSi Hehe
Alex Dimakis@AlexGDimakis@giffmana @DimitrisPapail @ChengleiSi Lucas, in a world of commodity models and scaled slop, a boutique research labs proposes something more deliciously bold: Think of Mozambique cashmere agents, asymmetrical overall environments and locally sourced world model custom losses.
#56Dan Roy@ROYDANROY@DimitrisPapail Cool stuff. I'd like to see a JEPA version though. A lot of the output of terminals is not useful to predict _most of the time_. You could at least speed up learning if you abstracted away some of the detail.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@DimitrisPapail This work is so cool as always @DimitrisPapail , and you are too kind!!
Dimitris Papailiopoulos@DimitrisPapailI'm just glad we did this before @lateinteraction and his amazing students :p
#172Alex Dimakis@ALEXGDIMAKISImprove your agents with one weird trick: ECHO says, when you SFT an agent, do not train it to predict only the agent replies, but also the terminal responses. When you GRPO, you use the same rollout to predict the terminal responses with cross entropy loss. Its basically free and gets extra supervision from the CLI. This apparently helps the model develop a 'world model' of the terminal, and improves performance, which was very surprising to me.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@giffmana @DimitrisPapail @ChengleiSi Lucas, in a world of commodity models and scaled slop, a boutique research labs proposes something more deliciously bold: Think of Mozambique cashmere agents, asymmetrical overall environments and locally sourced world model custom losses.
Lucas Beyer (bl16)@giffmana@DimitrisPapail @ChengleiSi This is nice work, so sorry for distracting from the substance, but I'm genuinely curious: > This work was done at AI Frontiers, a boutique research lab inside Microsoft Research. What does "boutique research lab" mean here?
#185Eric Horvitz@ERICHORVITZExciting results & direction with learning from signals from the environment—with implications for continual learning about the world. @DimitrisPapail @MSFTResearch
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#197Dimitris Papailiopoulos@DIMITRISPAPAIL@NovaSkyAI here's a simple skyRL patch to train better CLI agents, for free
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@giffmana @ChengleiSi A small group, of talented people, that are given free space to explore ideas that matter in the broader scope of AI, and specifically the area of computer use agents, but don't cost 1M to test :)
Lucas Beyer (bl16)@giffmana@DimitrisPapail @ChengleiSi This is nice work, so sorry for distracting from the substance, but I'm genuinely curious: > This work was done at AI Frontiers, a boutique research lab inside Microsoft Research. What does "boutique research lab" mean here?
@giffmana @ChengleiSi I came up with the phrasing, because it reminds me of how I'd describe with two words DM in its early days. One can only hope to come approximately close to that intellectual and technical space.
Dimitris Papailiopoulos@DimitrisPapail@giffmana @ChengleiSi A small group, of talented people, that are given free space to explore ideas that matter in the broader scope of AI, and specifically the area of computer use agents, but don't cost 1M to test :)
@giffmana @ChengleiSi also thanks for reading up to that part :D i know you have ton of cool stuff to work on today, so I'm grateful for your time.
Dimitris Papailiopoulos@DimitrisPapail@giffmana @ChengleiSi I came up with the phrasing, because it reminds me of how I'd describe with two words DM in its early days. One can only hope to come approximately close to that intellectual and technical space.
I'm just glad we did this before @lateinteraction and his amazing students :p
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@AlexGDimakis @giffmana @ChengleiSi lol
Alex Dimakis@AlexGDimakis@giffmana @DimitrisPapail @ChengleiSi Lucas, in a world of commodity models and scaled slop, a boutique research labs proposes something more deliciously bold: Think of Mozambique cashmere agents, asymmetrical overall environments and locally sourced world model custom losses.
@roydanroy I agree! We have some thoughts but they are related to compaction rather than Jepa.
Dan Roy@roydanroy@DimitrisPapail Cool stuff. I'd like to see a JEPA version though. A lot of the output of terminals is not useful to predict _most of the time_. You could at least speed up learning if you abstracted away some of the detail.
Turns out training your agent to be a world simulator improves its accuracy of solving problems
Yifu Qiu@ICLR 2026@yifuqiu98Internalizing world modeling as a native ability for agents.
Very rarely you stumble on a method that's simple, obvious in hindsight, free, and touches on every problem you care about: CLI agents, continual learning, self-improvement, world models.
ECHO is one of those
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@willccbb Humbled by the kind words. I also agree, it's bitter pilled AF
will brown@willccbbgod what a beautiful objective. i wonder how general you can push this. best non-distillation answer ive seen for knowledge acq during RL, feels bitter-pilled in a way that most self-teaching methods aren’t.
@willccbb yup! can also work in the absence of the normal GRPO loss which is also kinda nuts (as long as your tasks and current model are in some sense rich). I have no freaking clue what the ceiling is here
will brown@willccbb@DimitrisPapail i’d given up on the idea of using the rollout env tokens directly but had always still had the adv term in there (which doesn’t work, for reasons i now understand better). but dropping it makes so much more sense
@willccbb good question. will try it, makes a lot of sense.
will brown@willccbb@DimitrisPapail i’d be very curious to see it on small-taskset search with an efficiency bonus. does the model learn new facts and not need to search every time?
Lol you can continual learn by training on terminal outputs WITHOUT REWARDS
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
Prediction: by end of 2026 Echo will be part of standard agent RL trainers.
FREE LUNCH FOR EVERYONE
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
World modeling. Faster RL. Self-improvement without verifiers.
All from one extra loss term on your favorite open-weights CLI agent.
Happy Monday!
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@ChenhaoTan Thanks for checking out! I agree. You don't get too many of those in your career, so happy we stumbled upon it
Chenhao Tan@ChenhaoTanAlways a good sign that you are surprised that something has not been done before!
@willccbb I think we tried something like that last year or so 😅 so maybe open-source is way more than six months behind in some areas
will brown@willccbbgod what a beautiful objective. i wonder how general you can push this. best non-distillation answer ive seen for knowledge acq during RL, feels bitter-pilled in a way that most self-teaching methods aren’t.
#332John Langford@JOHNCLANGFORDA fun result: training to predict terminal output significantly accelerates RL for terminal agents.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#339will brown@WILLCCBBgod what a beautiful objective. i wonder how general you can push this. best non-distillation answer ive seen for knowledge acq during RL, feels bitter-pilled in a way that most self-teaching methods aren’t.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
a litmus test i’ve been thinking about for continual learning is bounding lifetime retrieval count per fact. a model should use tools to look things up, but gradually compound fuzzy memories of things they’ve searched, and eventually not need search. this could maybe work here
will brown@willccbbgod what a beautiful objective. i wonder how general you can push this. best non-distillation answer ive seen for knowledge acq during RL, feels bitter-pilled in a way that most self-teaching methods aren’t.
it also offers a clean bridge from pretraining to RL, which is another property i think we should expect from general continual learning methods. initially, everything is env tokens
will brown@willccbba litmus test i’ve been thinking about for continual learning is bounding lifetime retrieval count per fact. a model should use tools to look things up, but gradually compound fuzzy memories of things they’ve searched, and eventually not need search. this could maybe work here
@DimitrisPapail i’d given up on the idea of using the rollout env tokens directly but had always still had the adv term in there (which doesn’t work, for reasons i now understand better). but dropping it makes so much more sense
Dimitris Papailiopoulos@DimitrisPapail@willccbb Humbled by the kind words. I also agree, it's bitter pilled AF
@DimitrisPapail i’d be very curious to see it on small-taskset search with an efficiency bonus. does the model learn new facts and not need to search every time?
Dimitris Papailiopoulos@DimitrisPapail@willccbb yup! can also work in the absence of the normal GRPO loss which is also kinda nuts (as long as your tasks and current model are in some sense rich). I have no freaking clue what the ceiling is here
incredible Are we missing any other free, perfect, dense verifiers?
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#570Chenhao Tan@CHENHAOTANAlways a good sign that you are surprised that something has not been done before!
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#744Asli Celikyilmaz@REAL_ASLIHow do machines build a mental map of reality? 🧠
Check out this frontier investigation into *world models* from our team at @ms_aifrontiers. Proud to see @DimitrisPapail and colleagues pushing the boundaries of how we think about AI reasoning.
Dimitris Papailiopoulos@DimitrisPapailWorld modeling. Faster RL. Self-improvement without verifiers. All from one extra loss term on your favorite open-weights CLI agent. Happy Monday!
#878🎭@DEEPFATES@DimitrisPapail
Dimitris Papailiopoulos@DimitrisPapailWorld modeling. Faster RL. Self-improvement without verifiers. All from one extra loss term on your favorite open-weights CLI agent. Happy Monday!
@DimitrisPapail great work @DimitrisPapail
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#1776Super Dario@INDUCTIONHEADSWonderful. The terminal is the world to an agent. It learns to model the world
Dimitris Papailiopoulos@DimitrisPapailVery rarely you stumble on a method that's simple, obvious in hindsight, free, and touches on every problem you care about: CLI agents, continual learning, self-improvement, world models. ECHO is one of those
#1776Super Dario@INDUCTIONHEADSFYI, I will bet my last nickel this is part of Amthropics secret sauce
Super Dario@inductionheadsWonderful. The terminal is the world to an agent. It learns to model the world
