kalomaze warns that merging post-training processes kills true base models

kalomaze warns that merging post-training processes kills true base models · Digg

/TECH

To counter this, kache proposes releasing all checkpoints and training batches.

Combined views

16.7K

3 posts, first seen 6h ago

173 likes

To counter this, kache proposes releasing all checkpoints and training batches.

Commentary on X

dhruv bhatia@dhruvbhatia0

@kalomaze prob not intentional, but no base cripples other model training companies. Post post training the model is so unwilling to learn OOD tasks, you get terrible loss curves.

4:16 PM · Jul 15, 2026

Oli Wilkins@okwilkins

@kalomaze Would love for open data to be normalised also. It then can truly be called open source. @allen_ai leads the way here!

2:41 PM · Jul 15, 2026

Ben (no treats)@andersonbcdefg

@kalomaze blame mistral for being the first to start mixing instruct data into pretraining to juice benchmarks lol

4:48 PM · Jul 15, 2026

kalomaze@kalomaze

an unfortunate trend in modern open weights is the reluctance to release true bases in some cases this is arguably because there was no internal separation of concerns, i.e post training is more and more so being considered as a later part of the same overarching training process

2:35 PM · Jul 15, 2026

Combined views

16.7K

3 posts, first seen 6h ago

173 likes12 comments

12 comments

15 saves

2 reposts

Sentiment

Positive33.3%66.7%Negative

Positive Read

Negative users criticized reluctance to release true base models as it cripples other companies' training and involves mixing instruct data to juice benchmarks, while some advocated for normalizing open data.

Based on 3 visible X reactions from 9 accounts; directional sample.

Digg Deeper

Ask a question below.

Published answers will appear here.