Teortaxes spun a satirical scenario in which Mistral secretly overtrained a 256B-parameter model on 8T tokens for 128 epochs, naming the supposed result Mistral Le Foie Gras and its grokking effect LeStéatose as a nod to fatty liver.
I know y'all have been dazzled by Le Chaton Fat, but you've missed the leak about their secret project Mistral Le Foie Gras. It's an experiment in overtraining a 4T 256B LLM with 128 epochs of 8T tokens… Which has resulted in LeStéatose (known in the US as "grokking") stay alert
OMG THEY ARE DROPPING IT! ANTHROPIC IS SCREWED!
Scores higher than Fable in every benchmark!!
The post deliberately echoes a parallel circulating hoax about another fictional model called Le Chaton Fat, both framed as insider scoops that readers are told to treat with suspicion.
A reply from deepfates noted the approach could theoretically work, yet no evidence has appeared that Mistral ran any such experiment or released a matching model.
Positive users praise Mistral's active development of a 256B model for grokking while negative users mock the oversized parameters as unhealthy over-engineering.
@teortaxesTex *** The user might be testing us, one quadrillion tokens would require an amount of compute that even leading US labs might have trouble with. But Wait... the user is also known for making sarcastic posts very often, I'll answer briefly with a question. *** 1Q tokens?
@teortaxesTex I believe this would work
I know y'all have been dazzled by Le Chaton Fat, but you've missed the leak about their secret project Mistral Le Foie Gras. It's an experiment in overtraining a 4T 256B LLM with 128 epochs of 8T tokens… Which has resulted in LeStéatose (known in the US as "grokking") stay alert
@Pehdrew_ fo sure
@teortaxesTex another model?? man, Mistral is really cooking
@teortaxesTex The grokking process:
@teortaxesTex 这参数规模简直是模型界的脂肪肝预警
@teortaxesTex rumor says interns made it
@teortaxesTex You over-engineered the joke.
@teortaxesTex