ArXiv paper 'From Entropy to Epiplexity' prompts X discussion on why information theory lacks explanatory power and testable predictions for AI systems
Replies cite noise and unreliable estimators as practical ML barriers.
Story Brief
Replies cite noise and unreliable estimators as practical ML barriers.
Related Stories
3 linkedFrom Entropy to Epiplexity: Rethinking Information for...Can we learn more from data than existed in the generating process itself? Can new and useful information be constructed from merely applying deterministic transformations to existing data? Can...45d agoarXiv.org
Deep Learning is Not So Mysterious or DifferentDeep neural networks are often seen as different from other model classes by defying conventional notions of generalization. Popular examples of anomalous generalization behaviour include benign...45d agoarXiv.org
Compute-Optimal LLMs Provably Generalize Better With Scale
Deep Learning is Not So Mysterious or DifferentDeep neural networks are often seen as different from other model classes by defying conventional notions of generalization. Popular examples of anomalous generalization behaviour include benign...45d agoarXiv.orgCompute-Optimal LLMs Provably Generalize Better With Scale
