/Tech1h ago

@yacineMTB argues standard PPO with a shared trunk implicitly trains a world model, making dedicated architectures redundant

Story Overview

Research engineer yacineMTB is poking at the recent revival of interest in explicit world models for RL by noting that standard PPO already builds functional equivalents inside its shared actor-critic trunk. When an RNN is part of that trunk, hidden states end up encoding environmental properties that linear probes can readily surface, which he says renders many dedicated architectures more of a naming exercise than a technical necessity.

5301175.3K

#403

Original post

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex#501inTech

> our recent paper well, there have been other papers in this vein, but always nice to see someone still doing this… Just realized that I really want to see @SchmidhuberAI on @dwarkesh_sp's podcast.

Ravid Shwartz Ziv@ziv_ravid

1/ On Training in Imagination - Dwarkesh's episode has a segment on dreaming as one of the next training paradigms. The idea is that a model learns mostly inside its own, by imagining what would happen, instead of trying out for real.

We have a recent paper on exactly this 🥳🥳🥳

5:01 AM · Jul 2, 2026 · 4.2K Views

Developer Impact

Why practitioners keep reaching for plain PPO

yacineMTB's experience suggests the shared-trunk approach works well enough in practice that extra machinery often adds overhead rather than capability. He repeatedly encourages more people to simply train RNNs with PPO instead of chasing separate generative models.

Open Question

Where the implicit-versus-explicit comparison still sits

No direct head-to-head results appear in the thread, so it remains unclear how closely the PPO hidden states match the predictive and generative fidelity emphasized in the 2018 World Models paper. The discussion leaves open whether certain tasks still benefit from explicit imagination training.

Sentiment

Positive users welcome revival of the 2018 World Models paper for clarifying AI imagination via examples like Google's hallucinations, while negative users dismiss the debate as nomenclature memes and rebranding that feels sad.

Pos

50.0%

Neg

50.0%

2 comments with sentiment.

Cluster Engagement

Digg Deeper

No Digg Deeper questions have been answered for this story yet.

Posts from X

Most Activity

VIEWS995LIKES4

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex

It makes so much sense, Google even dreams/hallucinates it!

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex

1h99540

BOOKMARKS2REPLIES2

kache@yacineMTB

@teortaxesTex @SchmidhuberAI @dwarkesh_sp I never understood this world model meme. PPO with a shared actor critic trunk implicitly trains a world model.. it works incredibly well, When you have it train and RNN you can elicit properties about the world that it's in by training a linear probe off of the hidden state

Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTex

30m72632

kache@yacineMTB

@teortaxesTex @SchmidhuberAI @dwarkesh_sp A lot of these things are just nomenclature memes. Memes about things that I've already been happening in practice. It makes me very sad to see. More people should train RNNs with PPO

kache@yacineMTB

29m32821

Necromancer@jinwoo33x

The timeline loves rebranding old patches as revolutionary features.

Training an RNN inside the actor-critic loop was already letting the agent probe hidden state for world properties.

I’m excited we’re finally talking about it again, but sad it had to wear a fresh meme costume first.😗

15m101

Chudweed@chudweed

@teortaxesTex @SchmidhuberAI @dwarkesh_sp Schmidhuber!

31m15