Dimitris Papailiopoulos from Microsoft Research shares ECHO method for training CLI agents with environment prediction loss
Dimitris Papailiopoulos from Microsoft Research AI Frontiers posted results on ECHO, which adds an environment prediction loss to standard GRPO training for command-line agents. The method trains on both agent actions and terminal responses in one rollout and forward pass instead of masking outputs. It delivers improved benchmark scores across Qwen3 models. Researcher John Langford noted that forecasting terminal command outputs accelerates reinforcement learning for agents operating in command-line environments.
@DimitrisPapail @ChengleiSi This is nice work, so sorry for distracting from the substance, but I'm genuinely curious:
> This work was done at AI Frontiers, a boutique research lab inside Microsoft Research.
What does "boutique research lab" mean here?
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@AlexGDimakis @DimitrisPapail @ChengleiSi Hehe
Alex Dimakis@AlexGDimakis@giffmana @DimitrisPapail @ChengleiSi Lucas, in a world of commodity models and scaled slop, a boutique research labs proposes something more deliciously bold: Think of Mozambique cashmere agents, asymmetrical overall environments and locally sourced world model custom losses.
@DimitrisPapail This work is so cool as always @DimitrisPapail , and you are too kind!!
Dimitris Papailiopoulos@DimitrisPapailI'm just glad we did this before @lateinteraction and his amazing students :p
#172Alex Dimakis@ALEXGDIMAKISImprove your agents with one weird trick: ECHO says, when you SFT an agent, do not train it to predict only the agent replies, but also the terminal responses. When you GRPO, you use the same rollout to predict the terminal responses with cross entropy loss. Its basically free and gets extra supervision from the CLI. This apparently helps the model develop a 'world model' of the terminal, and improves performance, which was very surprising to me.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@giffmana @DimitrisPapail @ChengleiSi Lucas, in a world of commodity models and scaled slop, a boutique research labs proposes something more deliciously bold: Think of Mozambique cashmere agents, asymmetrical overall environments and locally sourced world model custom losses.
Lucas Beyer (bl16)@giffmana@DimitrisPapail @ChengleiSi This is nice work, so sorry for distracting from the substance, but I'm genuinely curious: > This work was done at AI Frontiers, a boutique research lab inside Microsoft Research. What does "boutique research lab" mean here?
#197Dimitris Papailiopoulos@DIMITRISPAPAIL@NovaSkyAI here's a simple skyRL patch to train better CLI agents, for free
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@giffmana @ChengleiSi A small group, of talented people, that are given free space to explore ideas that matter in the broader scope of AI, and specifically the area of computer use agents, but don't cost 1M to test :)
Lucas Beyer (bl16)@giffmana@DimitrisPapail @ChengleiSi This is nice work, so sorry for distracting from the substance, but I'm genuinely curious: > This work was done at AI Frontiers, a boutique research lab inside Microsoft Research. What does "boutique research lab" mean here?
@giffmana @ChengleiSi I came up with the phrasing, because it reminds me of how I'd describe with two words DM in its early days. One can only hope to come approximately close to that intellectual and technical space.
Dimitris Papailiopoulos@DimitrisPapail@giffmana @ChengleiSi A small group, of talented people, that are given free space to explore ideas that matter in the broader scope of AI, and specifically the area of computer use agents, but don't cost 1M to test :)
@giffmana @ChengleiSi also thanks for reading up to that part :D i know you have ton of cool stuff to work on today, so I'm grateful for your time.
Dimitris Papailiopoulos@DimitrisPapail@giffmana @ChengleiSi I came up with the phrasing, because it reminds me of how I'd describe with two words DM in its early days. One can only hope to come approximately close to that intellectual and technical space.
I'm just glad we did this before @lateinteraction and his amazing students :p
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@AlexGDimakis @giffmana @ChengleiSi lol
Alex Dimakis@AlexGDimakis@giffmana @DimitrisPapail @ChengleiSi Lucas, in a world of commodity models and scaled slop, a boutique research labs proposes something more deliciously bold: Think of Mozambique cashmere agents, asymmetrical overall environments and locally sourced world model custom losses.
Turns out training your agent to be a world simulator improves its accuracy of solving problems
Yifu Qiu@ICLR 2026@yifuqiu98Internalizing world modeling as a native ability for agents.
Very rarely you stumble on a method that's simple, obvious in hindsight, free, and touches on every problem you care about: CLI agents, continual learning, self-improvement, world models.
ECHO is one of those
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
Lol you can continual learn by training on terminal outputs WITHOUT REWARDS
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
Prediction: by end of 2026 Echo will be part of standard agent RL trainers.
FREE LUNCH FOR EVERYONE
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
World modeling. Faster RL. Self-improvement without verifiers.
All from one extra loss term on your favorite open-weights CLI agent.
Happy Monday!
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
@ChenhaoTan Thanks for checking out! I agree. You don't get too many of those in your career, so happy we stumbled upon it
Chenhao Tan@ChenhaoTanAlways a good sign that you are surprised that something has not been done before!
#332John Langford@JOHNCLANGFORDA fun result: training to predict terminal output significantly accelerates RL for terminal agents.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#339will brown@WILLCCBBgod what a beautiful objective. i wonder how general you can push this. best non-distillation answer ive seen for knowledge acq during RL, feels bitter-pilled in a way that most self-teaching methods aren’t.
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
incredible Are we missing any other free, perfect, dense verifiers?
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#570Chenhao Tan@CHENHAOTANAlways a good sign that you are surprised that something has not been done before!
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#744Asli Celikyilmaz@REAL_ASLIHow do machines build a mental map of reality? 🧠
Check out this frontier investigation into *world models* from our team at @ms_aifrontiers. Proud to see @DimitrisPapail and colleagues pushing the boundaries of how we think about AI reasoning.
Dimitris Papailiopoulos@DimitrisPapailWorld modeling. Faster RL. Self-improvement without verifiers. All from one extra loss term on your favorite open-weights CLI agent. Happy Monday!
#878🎭@DEEPFATES@DimitrisPapail
Dimitris Papailiopoulos@DimitrisPapailWorld modeling. Faster RL. Self-improvement without verifiers. All from one extra loss term on your favorite open-weights CLI agent. Happy Monday!
@DimitrisPapail great work @DimitrisPapail
Dimitris Papailiopoulos@DimitrisPapailhttp://x.com/i/article/2056344151235387392
#1776Super Dario@INDUCTIONHEADSWonderful. The terminal is the world to an agent. It learns to model the world
Dimitris Papailiopoulos@DimitrisPapailVery rarely you stumble on a method that's simple, obvious in hindsight, free, and touches on every problem you care about: CLI agents, continual learning, self-improvement, world models. ECHO is one of those
#1776Super Dario@INDUCTIONHEADSFYI, I will bet my last nickel this is part of Amthropics secret sauce
Super Dario@inductionheadsWonderful. The terminal is the world to an agent. It learns to model the world
