@YiMaTweets @burny_tech This was partly written with AI assistance, but the information we want to keep (during lossy compression) is empirically grounded.
Reality as Corrective: Physical Grounding as a Unified Solution to AI Alignment Pathologies SSRN.6397178 http://dx.doi.org/10.2139/ssrn.6397178
@YiMaTweets Interesting perspective! I wonder if there is a trade-off between efficiency and generality here. Should a good representation capture all variation in the data, or be more task-guided, preserving only useful factors while compressing away irrelevant random variation?
I think compression is not the goal of intelligence. It is a means of transforming the world into a usable representation in order to reduce uncertainty.
Human language works in a similar way. We cannot pass the cognitive field before language directly to another person. So the speaker translates and compresses thoughts, assumptions, perspectives, and evaluative axes into words.
The listener does not simply store those words as they are. They reconstruct them within their own cognitive field.
In that sense, good compression is not just about reducing the amount of information. It is about preserving a structure that can later be unfolded into meaning, prediction, judgment, and action.
Perhaps intelligence is both the ability to compress and the ability to reconstruct the world from compressed representations.