Judea Pearl argues that the distinction between explicit and implicit storage explains computational variation without needing complex entropy paradoxes

@yudapearl @bishopb0b @leecronin @Sara_Imari They don’t consider computation which means results like the DPI and the symmetry of information come up empty handed in explaining the success of synthetic data, pseudorandom numbers, ordering, emergence. We can shed light on these behaviours by accounting for computation.

@yudapearl @bishopb0b @leecronin @Sara_Imari Time bounded entropy violates the DPI, and the symmetry of information. When we target predictable information that a time bounded learner can extract from data, emergence, systems like alphazero, pseudorandom numbers, OOD generalization become more understandable.