Why does predicting in latent space work well on messy data?

A post asks why approaches such as JEPA, CPC and SimCLR work well on messy data with changing lighting, camera angles and busy backgrounds. The author says the usual explanation is that these methods can ignore nuisance factors, but argues that this leaves a conundrum.