Meta AI's Lucas Beyer argues pre-training on massive video data is the best long-term pathway for learning physics

For data efficiency I'm 50/50, because i can see both of these: 1) a model that can learn the physics relevant for a specific problem from not-massive data of that problem 2) a model that learns ALL the physics from the infinite amount of videos, then fine-tune per problem on little problem-specific data.

I can totally see 1 happening, but if i had to bet on the long term, i would bet all my eggs on 2, and even that 2 will be more fine-tuning data-efficient than 1.