OpenAI's Gabriel Petersson argues that human language is too compressed to convey highly complex concepts to advanced AI
Investor Alex Rampell agreed, calling human-to-AI communication highly lossy.
Positive users agree lossy language makes explaining concepts harder for smarter AI models as communication becomes the bottleneck, while negative users dismiss it as a nonexistent problem or skill issue.
No Digg Deeper questions have been answered for this story yet.
Most Activity
@gabriel1 Thought -> language -> model is very lossy compression indeed
words are very lossy pointers to complex concepts in our brains
explaining these concepts to ai become incrementally harder as models become smarter and can do more things
@gabriel1 We could try direct latent space communication in COCONUT style. Might be hard to do for humans -> LLM, due to engineering, but we could still test it by doing multi agent LLM -> LLM communication like this first. Just talk in d_model sized vectors without projecting to tokens.
@gabriel1 they are already lossy for the concepts we can name, but then there are 10x more concepts that we know but dont have words for
This is why I keep thinking the prompt box is a bad default.
When the concept is in your head, words already compress it. Then the model has to reconstruct the missing shape from the compressed version.
A screenshot, sketch, messy folder, or camera frame often carries the part language drops.
@gabriel1 yes. and there are solutions here.
@gabriel1 And tokens are very complex pointers to lossy concepts in the model weights.
./nod
words are like the surface of the bubble latent thought is like the inside of the bubble of semantic meaning and is murky superposition you can travel
Words are great, bc they're hooky. They're the outcroppings you can chain and climb in interesting ways to move around the bubble with intent. However they lose something when you go from the many to one crush from the inside to the discrete (words) outside :3 So interesting to think of the trade offs and how they differ 🤔🧐
I disagree. The models do not become smarter. Rather, they become more complex. Therefore, the method of supplying them with data must change. Neural networks succeed in programming, drones, and agriculture. They fail in perception related to emotions or psychological analysis, and in movement or fully conceptualizing reality. Perhaps the solution is what Yann LeCun said: that the model learns from its surroundings. But this solution is illogical; if used, it won't work because of the enormous amount of information. The solution is for artificial intelligence to have a body that enables it to know its limits and what it truly needs to learn in a genuine way. The solution is Tesla Optimus robots; they will be the alternative when Elon Musk stops the discrimination posts and turns seriously to the Tesla lab.
@gabriel1 we need brain-to-codex interface
@gabriel1 I speak to the models in binary
@gabriel1 Yes. Now elevate this to nesim. What is the world you see like? What’s Illya world gabriel?
@gabriel1 that's why we have expressive typing in some programming languages like Rust. It is crazy that you are to solve a non existing problems !
@gabriel1 You need to read on certainty by Wittgenstein
@gabriel1 what do you mean harder the way fable grasped my intuition was buttery smooth. Feels like its only gotten easier to explain my thoughts to frontier models
@gabriel1 Agree, and like the way you worded this. What medium has higher signal?
@gabriel1 you can’t connect directly your brain with a high throughput capacity channel communication to a computer, but you could cache your brain into an ai that speaks for you, maybe
@gabriel1 Opposite day
@gabriel1 Nah, you need to think and explain it in simpler words, and so do the LLMs....
@gabriel1 do you think the fix is better prompting languages, or models that learn your personal concept-space over time?
@gabriel1 the models are getting smarter, but the humans that are prompting have the same level of intelligence, soon we will need something that will translate our bare bone prompts to models but in more precise ways