Dimitris Papailiopoulos says information theory was not designed to address algorithmic phenomena in AI and instead analyzes limits on communication, storage, retrieval, and compression
The exchange began with Jiaxin Wen claiming the framework cannot explain AI.
Information theory was not built to explain algorithmic phenomena. It's a beautiful framework for arguing about the limits of information: what can be communicated, stored, retrieved, compressed etc. Most attempts I've seen at forcing IT onto AI feel like trying to make coffee with a katana. Magnificent instrument but wrong job :)
Jiaxin Wen@jiaxinwen22It's very disappointing that information theory cannot explain AI at all.
#263Andrew Carr 🤸@ANDREW_N_CARRme after reading the epiplexity paper
Jiaxin Wen@jiaxinwen22It's very disappointing that information theory cannot explain AI at all.
@DimitrisPapail @jiaxinwen22 Beside Stefano Ermon work...
Dimitris Papailiopoulos@DimitrisPapailInformation theory was not built to explain algorithmic phenomena. It's a beautiful framework for arguing about the limits of information: what can be communicated, stored, retrieved, compressed etc. Most attempts I've seen at forcing IT onto AI feel like trying to make coffee with a katana. Magnificent instrument but wrong job :)
@jiaxinwen22 What field can?
Jiaxin Wen@jiaxinwen22It's very disappointing that information theory cannot explain AI at all.
@DimitrisPapail ai folks somehow are still frequently using description length or proposing new variants
Dimitris Papailiopoulos@DimitrisPapailInformation theory was not built to explain algorithmic phenomena. It's a beautiful framework for arguing about the limits of information: what can be communicated, stored, retrieved, compressed etc. Most attempts I've seen at forcing IT onto AI feel like trying to make coffee with a katana. Magnificent instrument but wrong job :)
@DimitrisPapail I am always impressed by their papers but feel disappointed when using those metrics to explain AI
Dimitris Papailiopoulos@DimitrisPapailInformation theory was not built to explain algorithmic phenomena. It's a beautiful framework for arguing about the limits of information: what can be communicated, stored, retrieved, compressed etc. Most attempts I've seen at forcing IT onto AI feel like trying to make coffee with a katana. Magnificent instrument but wrong job :)