/Tech1h ago

Nathan Lambert Says Pretraining Data Cleanup With Fable Not Worth Cost

465022.3K

#80

Original post

Nathan Lambert@natolambert#80inTech

@jxmnop I’m sorry sir but looking at every row of pretraining data with fable is not worth the money

Jack Morris@jxmnop

An underrated part of this discussion is that (a) there's huge leverage in improving data, and (b) there's no way Anthropic could safeguard this

xAI could instruct Fable to look through EVERY row of pretraining data and fix any typos and errors. this probably the single highest-leverage activity for a lab playing catchup

and it's not possible for Anthropic to prevent this without completely kneecapping the model itself, because data quality work looks like any other kind of knowledge work ("check this text for errors", "rewrite this in a formal tone")

11:39 AM · Jun 12, 2026 · 1.8K Views

Sentiment

Positive users agree with Nathan Lambert that pretraining data cleanup with Fable is not worth the cost versus other high-leverage options, while negative users challenge prior claims it was the highest-leverage improvement.

Pos

50.0%

Neg

50.0%

2 comments with sentiment.

Cluster Engagement

Posts from X

Most Activity

VIEWS433BOOKMARKS1LIKES21REPLIES1

Nathan Lambert@natolambert

@jxmnop You said it was the highest leverage thing to do lol

Jack Morris@jxmnop

@natolambert obviously, but u are missing the point

i'm saying most of what makes models better is DATA WORK: evals, rubrics, error analysis, and so on

Fable can do this, Mythos will do this, future models will do this

1h433211

Jack Morris@jxmnop

@natolambert obviously, but u are missing the point

i'm saying most of what makes models better is DATA WORK: evals, rubrics, error analysis, and so on

Fable can do this, Mythos will do this, future models will do this

1h3016

renji@brickroad7

@natolambert @jxmnop Really??

1h26

Ian@dumbfook

@natolambert @jxmnop im sure there are other parts that are pretty easy, pretty cheap and super high leverage but I still agree with your perspective more

1h25