We ground discussion in the history and philosophy of science. What did it take for other fields to move from cataloging phenomena to predicting and controlling them? AI can learn from that playbook.
Post hoc analysis can certainly be useful, especially if you’re primarily concerned with the behavior of a specific deployed model. But looking at a static model will not tell you why the model developed a behavior. The real causal story must go back to the training process.