High Dimensions Turn Boundary Generalization Into Volume Advantage
Tweet from Tsinghua PhD student explains volume advantage for generalization in high-dimensional spaces.
TLDR
You Jiacheng posted that generalization and reinforcement learning operate on boundaries, which appears problematic. He noted that in high-dimensional space the boundary occupies most of the volume. The post gave an example of training on 10k samples from a unit ball and generalizing to a (1+δ) ball that contains at least (1+δ)^d times 10k cases. The message frames high dimension as a feature rather than a drawback for such tasks.
Combined views
6.6K
1 Source, first seen 32d ago