So
pre-training → CPT/mid-training → SFT → {RL, many experts} → MOPD warm-up / SFT → MOPD → maybe loop a few times
This is starting to look like the pipelines we had before deep learning.
2:05 PM · Jun 5, 2026 · 10.6K Views
Dhruv Batra, Yutori AI co-founder, argues AI training pipelines are starting to resemble pre-deep learning multi-stage workflows
Markus Wulfmeier compares this complexity to Kuhn's paradigm shifts.
8158109012.1K
Sentiment
Users affirm the growing complexity of AI training pipelines with multi-stage post-training loops, recognizing parallels to elaborate older NLP setups that still underfit.
Pos
100.0%
Neg
0.0%
2 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS916BOOKMARKS2LIKES17RETWEETS3
Zero Void@0x00_void
@DhruvBatra_ we deleted feature engineering only to invent training engineering
Dhruv Batra@DhruvBatra_
So
pre-training → CPT/mid-training → SFT → {RL, many experts} → MOPD warm-up / SFT → MOPD → maybe loop a few times
This is starting to look like the pipelines we had before deep learning.
18hViews 916Likes 17Bookmarks 2
REPLIES1
Markus Wulfmeier@m_wulfmeier
Not an argument for it but the current cycle between simplicity and complexity feels quite natural. When squinting, this is Kuhn's argument around progress in science and Gabriel's for software.
New paradigm enables and even forces simplicity, because it works well in its pure form and because we don't understand it well enough yet to improve it with complex variants.
Over time improvements become more specific (to enable short term gains) and harder to build upon due to complexity correlations across all previous variants.
We get fed up with complexity, improvements are slower, and after a while we find a new paradigm...
Dhruv Batra@DhruvBatra_
So
pre-training → CPT/mid-training → SFT → {RL, many experts} → MOPD warm-up / SFT → MOPD → maybe loop a few times
This is starting to look like the pipelines we had before deep learning.
8hViews 659Likes 3Bookmarks 0
Dhruv Batra@DhruvBatra_
@0x00_void Amen!
17hViews 376Likes 1
Velon@velonxbt
@DhruvBatra_ real recognize real. those old NLP pipelines had more stages than a Marvel movie and still underfit.
18hViews 346
Markus Wulfmeier@m_wulfmeier
@DhruvBatra_ It's also a very human thing in general. I wonder if there's work relating it to organizations and political systems..
7hViews 3