François Fleuret positions LLMs as samplers from the probability density of ideas produced by the most innovative humans, which opens a path for outputs that feel new even if the underlying training never explicitly rewards originality.
i’m really surprised that people don’t see this.
It’s mathematically true that llms can’t come up with novel ideas, because the whole point of training is to reduce loss, gain rewards so that the model adhere to rules and ground truth.
if you have a model that can come up with novel ideas, it must have high loss during sft or rl.
Critics point to loss minimization and reward maximization as forces that keep models tethered to existing ground-truth patterns, leaving little room for anything mathematically outside the training distribution.
AI researcher Lucas Beyer labeled the no-novelty argument top-tier bait and shared a Renaissance-style image mocking the idea as an obvious hook, signaling that density-modeling views still have vocal defenders in the field.
Users debate claims that LLMs cannot generate novel ideas due to training data, with positive replies noting creative prompts produce original outputs while negative ones dismiss them as automated plagiarism for thieves.
No Digg Deeper questions have been answered for this story yet.
The density they are learning is that of people who had new ideas.
i’m really surprised that people don’t see this.
It’s mathematically true that llms can’t come up with novel ideas, because the whole point of training is to reduce loss, gain rewards so that the model adhere to rules and ground truth.
if you have a model that can come up with novel ideas, it must have high loss during sft or rl.
@giffmana They always come up with new ideas. If you use prompts which are not in the dataset you will get answers that are not in the dataset.
@giffmana so what novel ideas did llms came up with ? any invention worth mentioning? any technological breakthrough?
@francoisfleuret Isn’t the idea that, at large enough scale, models may gain emergent capabilities by learning the process that generated the data, rather than merely matching points on a curve?
@francoisfleuret Oh, it is just automated plagiarism Francois. That's why the stupidest people love it, because they are stupid thieves. :)
@francoisfleuret Ideas are made of other ideas
@francoisfleuret a skill is a statistical regularity