@natolambert obviously, but u are missing the point
i'm saying most of what makes models better is DATA WORK: evals, rubrics, error analysis, and so on
Fable can do this, Mythos will do this, future models will do this
@jxmnop I’m sorry sir but looking at every row of pretraining data with fable is not worth the money
to clarify, looping Fable over huge datasets wouldn't be a good use of compute for many reasons
but much of what goes into better models is DATA WORK: eval design, rubrics, error analysis, repairing noisy data
Fable can do this! and Mythos will do this too
An underrated part of this discussion is that (a) there's huge leverage in improving data, and (b) there's no way Anthropic could safeguard this
xAI could instruct Fable to look through EVERY row of pretraining data and fix any typos and errors. this probably the single highest-leverage activity for a lab playing catchup
and it's not possible for Anthropic to prevent this without completely kneecapping the model itself, because data quality work looks like any other kind of knowledge work ("check this text for errors", "rewrite this in a formal tone")
@jxmnop You said it was the highest leverage thing to do lol
@jxmnop data workers sweating every time someone calls data work compute-free