Yun-Ta Tsai's breakdown reveals that real-world machine learning work is dominated by the unglamorous grind of checking outputs and scrubbing inputs, leaving almost no room for the flashy training step that outsiders imagine drives everything.
So true
Many people think any given ML project is 99% training.
In reality, it’s 50% evaluation, 40% data cleaning, 8% integration, and 2% training.
The first two set the noise floor for learning. No ML magic matters; the model cannot lower the noise floor, as that’s the optimal bound of Shannon encoding of your data.
Thus, not a single day goes by without me thinking about ontology. Even the old labels have to be constantly reviewed.
The 40 percent spent on cleaning and 50 percent on evaluation together define the performance ceiling; as Tsai notes via Shannon encoding, the model cannot magically reduce noise below what the data itself allows.
Tsai stresses that ontology work never ends, with even legacy annotations requiring constant re-examination to keep the noise floor from rising over time.
Many users agree with the Hugging Face CEO and Elon Musk that ML projects are mostly evaluation and data cleaning rather than model training, while a few reply with insults or dismissals.
Sheing Ng @sashimikun_void how many percentage of that 2% training is run by AI? educated guess
Nearly 100%. The 2% "training" slice is almost entirely the automated compute run on GPUs/TPUs once setup is done; human effort there is just launching and monitoring jobs. The rest of the project (data work and evaluation) stays human-heavy.
@elonmusk Data quality is still the highest leverage optimization in ML
@powertoold Even a genius needs a good textbook. An IQ of 180 cannot learn linear algebra if all you give is caveman drawing.
Many people think any given ML project is 99% training.
In reality, it’s 50% evaluation, 40% data cleaning, 8% integration, and 2% training.
The first two set the noise floor for learning. No ML magic matters; the model cannot lower the noise floor, as that’s the optimal bound of Shannon encoding of your data.
Thus, not a single day goes by without me thinking about ontology. Even the old labels have to be constantly reviewed.
@SheriffTouray45 Ontology is used to teach the model context. Wrong or vague ontology would force the model to learn the wrong reasoning.
@yunta_tsai 💯
@ccorcos Ontology should convey the objective. In RL, ontology is expressed in reward functions. In SFT, ontology is expressed in dense labels.
You have to look at them with a blank mind and ask yourself what you learned — no presumption or bias except a blank sheet of mind.
@yunta_tsai You're basically saying that ML has no intelligence, and you have to hold its hand through every example
@DagnySpeaks @powertoold You think Mozart has no teacher? A teacher does not mean you need someone one-on-one to hand-hold you, but someone to be inspired by.
He grew up in music rich family — soaking in it.
@elonmusk Please speak about the woke mind virus before it’s too late man.
@yunta_tsai What does 'thinking about ontology' mean practically in labeling data for vision/AI systems?
@elonmusk @Tesla_Optimus grinding 24/7 on the Moon and Mars with @starlink coverage. @elonmusk is a straight-up genius creator/builder/engineer. The only explanation? He’s got a whole squad of goblins working for him lol @elonmusk Aka Goblon. @starlink @imagine @grok @spacex
@elonmusk
@yunta_tsai data and evals >>>>>>>>>>> everything else 💯
@yunta_tsai How are you modeling ontology in this context?
@yunta_tsai data is really the programming language of AI
10yrs ago, I found the most efficient thing you can do is just deleting corrupted and poisoned edge case data from a big dataset.
Quite often the effort of evaluating and labelling data in the ambiguous region just isn’t worth it, people try and use a scalpel and create a very high fidelity representation from ambiguous stuff, but it’s almost always better to just delete all the fuzzy stuff.
If you have a lot of ambiguous fuzzy stuff then you usually need more dimensions (orthogonal) to de-fuzz the ambiguity.
@elonmusk Especially with so much trash information from the sources nowadays.
Top 10 In-Demand Programming Languages (2026)
1) Python - AI, Machine Learning, Data Science, Web Development 2) JavaScript - Web Development (Frontend & Backend) 3) Java - Enterprise Systems, Android Development 4) C# - Game Development, Enterprise Applications 5) Swift - iOS & Apple App Development 6) TypeScript - Scalable Web Applications 7) Kotlin - Modern Android Development 8) Ruby - Startups, Rapid Development 9) Rust - System Programming, High Performance & Security 10) Go (Golang) - Cloud Computing, Microservices, Scalability
Source: Crossover
@elonmusk Its unfair when i post "so, true" the @X algorithm shadow bans me but its ok with elon.
@geo_anima @SheriffTouray45 If your eval and data are extremely good, then you don’t really need a big model.
All these extra parameters are trying to compress the noise.
