/Tech2h ago

Researchers Urge Study of AI Training Dynamics Over Post-Hoc Analysis

341511.4K
Stella Biderman#218
Original post
Stella Biderman@BlancheMinerva#218inTech

Post hoc analysis can certainly be useful, especially if you’re primarily concerned with the behavior of a specific deployed model. But looking at a static model will not tell you why the model developed a behavior. The real causal story must go back to the training process.

Stella Biderman@BlancheMinerva

Models are not static objects. They're snapshots of time-evolving processes shaped by data, objectives, architectures, and optimization. But most research treats them as fixed artifacts, analyzing behaviors after training instead of asking why they emerged.

12:08 PM · Jun 10, 2026 · 410 Views
Sentiment

Many users are showing appreciation for the researchers' work on AI training dynamics and open problems in bias and memorization by thanking co-authors and planning to share the papers with their teams.

Pos
100.0%
Neg
0.0%
5 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS735BOOKMARKS1LIKES19RETWEETS5
Stella Biderman@BlancheMinerva

Read the full paper: https://arxiv.org/abs/2606.06533 or come listen to our oral @icmlconf!

Huge thanks to my co-authors @aflah02101 @niloofar_mire @linguist_cat @FazlBarez @nsaphra

Stay tuned for a related workshop (hopefully) at NeurIPS too!

Stella Biderman@BlancheMinerva

Part of why post hoc analysis dominates: it's the only thing most researchers CAN do. Almost no one releases intermediate checkpoints or training data. we built MultiBERT and Pythia to set a better standard, and it's been great to see work like OLMo and Marin follow our lead.

2hViews 735Likes 19Bookmarks 1
REPLIES1
Stella Biderman@BlancheMinerva

A common issue with position papers is that they leave the reader wondering “okay, but what should I actually do”? To address this we provide open problems on a wide variety of topics throughout to illustrate our perspectives and guide future research

Stella Biderman@BlancheMinerva

A test for progress: a science of AI should support progressively stronger forms of understanding: 1. Predict outcomes from early training signals 2. Intervene to correct trajectories on undesirable paths 3. Design training procedures that reliably produce desired properties

2hViews 260Likes 9Bookmarks 0
Stella Biderman@BlancheMinerva

We ground discussion in the history and philosophy of science. What did it take for other fields to move from cataloging phenomena to predicting and controlling them? AI can learn from that playbook.

2hViews 191Likes 10
Stella Biderman@BlancheMinerva

Part of why post hoc analysis dominates: it's the only thing most researchers CAN do. Almost no one releases intermediate checkpoints or training data. we built MultiBERT and Pythia to set a better standard, and it's been great to see work like OLMo and Marin follow our lead.

2hViews 313Likes 6
Stella Biderman@BlancheMinerva

A test for progress: a science of AI should support progressively stronger forms of understanding: 1. Predict outcomes from early training signals 2. Intervene to correct trajectories on undesirable paths 3. Design training procedures that reliably produce desired properties

2hViews 176Likes 6
Turing@turingcom

@BlancheMinerva @icmlconf @Aflah02101 @niloofar_mire @linguist_cat @FazlBarez @nsaphra Going to share this with the team!👏

2hViews 43Likes 1