god what a beautiful objective. i wonder how general you can push this. best non-distillation answer ive seen for knowledge acq during RL, feels bitter-pilled in a way that most self-teaching methods aren’t.

ECHO is now on arxiv. Please share your thoughts and comments!

Very rarely you stumble on a method that's simple, obvious in hindsight, free, and touches on every problem you care about: CLI agents, continual learning, self-improvement, world models.

ECHO is one of those

World modeling. Faster RL. Self-improvement without verifiers.

All from one extra loss term on your favorite open-weights CLI agent.

Happy Monday!

Lol you can continual learn by training on terminal outputs WITHOUT REWARDS

nice work by @DimitrisPapail and @VaishShrivas!

this work is reinforcing a recent trend that tries to make foundation models jointly predict future states (aka 'world models') and actions instead of actions alone.

we're seeing it in different forms, like World Action Models in embodied agents, or implicit world modeling in Early Experience (https://arxiv.org/abs/2510.08558). also some interesting link to on-policy self-distillation.

shared learning here is, there's still rich supervision signals that are underexplored. such signals were hard to exploit in classic ML, but foundation models have made it possible, potentially creating a recursive self-improvement loop.

incredible Are we missing any other free, perfect, dense verifiers?

Just realized ECHO fits a years long obsession of transformers and computers.

"Looped Transformers are Computers" "Can You Train a Transformer to be Computer?" And now "Can You Train a Transformer to Simulate a Computer?"

Blame my hobbyist love of theory of computation

Prediction: by end of 2026 Echo will be part of standard agent RL trainers.

FREE LUNCH FOR EVERYONE

Turns out training your agent to be a world simulator improves its accuracy of solving problems

they added a world modeling loss term to a CLI policy model and it just got better! this has been an increasingly popular trend in robotic policy training over the past year (e.g. cosmos-policy, dreamzero) and i love that it's catching on in the LLM area now. every feedback from the world holds bits of info worth learning, it's a shame most have only cared about the extremely sparse reward feedback and dismiss the rest.

Wonderful. The terminal is the world to an agent. It learns to model the world

Improve your agents with one weird trick: ECHO says, when you SFT an agent, do not train it to predict only the agent replies, but also the terminal responses. When you GRPO, you use the same rollout to predict the terminal responses with cross entropy loss. Its basically free and gets extra supervision from the CLI. This apparently helps the model develop a 'world model' of the terminal, and improves performance, which was very surprising to me.

One aspect that also appreciate about ECHO is that it can reduce reliance on SFT data to jump start a CLI agent.

An example: comparing with the OpenThoughts-Agent which is Qwen3-8B SFT’d on ∼15k GLM-4.6 trajectories, ECHO on base Qwen and NO SFT closes the gap.

Kinda cool!

a litmus test i’ve been thinking about for continual learning is bounding lifetime retrieval count per fact. a model should use tools to look things up, but gradually compound fuzzy memories of things they’ve searched, and eventually not need search. this could maybe work here

How do machines build a mental map of reality? 🧠

Check out this frontier investigation into *world models* from our team at @ms_aifrontiers. Proud to see @DimitrisPapail and colleagues pushing the boundaries of how we think about AI reasoning.

I'd bet stuff like this is most of the OS-closed gap, attention to detail based on well reasoned / high taste conceptual theories to catch all these various subtle flaws. The tail end of the distribution is where all the value is.

There's so many little things like this. Most of the ones I can think of are in pretraining bc I'm still learning RL but they're all just like this in spirit

Very clever

FYI, I will bet my last nickel this is part of Amthropics secret sauce

https://arxiv.org/abs/2605.24517