CoreAutoAI co-founder Rohan Anil criticizes Google's new "midtraining" unit, arguing the phase should simply be called training

what even is midtraining?

I am quite serious, for those thinking I am joking.

Mid training is just too late to learn good features particularly worse when networks have been deep fried.

@_aidan_clark_ No joke: when i saw this tweet earlier today was soooo close of digging up your tweet from a few years ago where you basically introduced the term to the world by saying "we're hiring for midtraining team"

Even funnier is Google now also recently has an Omni team!

you're telling me google's training is mid???

@_aidan_clark_ mid training

@_aidan_clark_ org chart issue

@_aidan_clark_ i am mid and i train

Jokes about mid training write themselves, but not by me.

@_aidan_clark_ training but mid

@_aidan_clark_ Skill issue + headcountmaxxxing issue

@_aidan_clark_ Actually, @grok can you find the tweet?

@_arohan_

@_aidan_clark_ isn't it just a pre-post-train, post-train

@_aidan_clark_ drop the mid. it’s cleaner

@giffmana shhhhhh shhhhh I like seeing everyone try to mansplain it

@_aidan_clark_ Fine-Tuning on SFX, the last training stage would be RL I guess.

@_aidan_clark_

@_aidan_clark_ extra training for when you're model is mid

@_arohan_ There not gonna close the gap it’s extremely wide ^ we already accepted that last year! They keep chasing hype and not paying attention that’s on them

Didn’t amazon coin this as a term after a base model checkpoint where you could still infuse domain specific knowledge and have it be able to blend proprietary and general knowledge into its weight before proper post training and RL?

Idk why they wouldn’t just say continued pre training like literally anyone else ever