The curriculum uses historical paradigm shifts to teach creative research.
If I rewrote the AGI safety fundamentals curriculum today most of it would be history and philosophy of science.
Learning about past paradigm shifts seems like the most scalable way to intuitively understand what deep, creative, principled research looks like.
Many users suggested additions like Popper, Feyerabend, and mechinterp case studies to the AGI safety curriculum because they see history and philosophy of science as directly relevant to alignment.
No Digg Deeper questions have been answered for this story yet.
A rough sketch might look like: - Kuhn on scientific revolutions - Strevens’ “coral reef” analogy for science - Garrabrant induction as a model of scientific epistemology - Schelling’s Strategy of Conflict - The Pascal/Fermat correspondence - Strohl on naturalism - Hasok Chang on inventing temperature - A case study of the construction of the heliocentric model - A case study of the construction of the concept of AGI - A case study of the concept of “generalization” in ML and AI safety - A case study of the idea of embedded agency - Perceptual control theory and active inference
Other suggestions along these lines?
If I rewrote the AGI safety fundamentals curriculum today most of it would be history and philosophy of science.
Learning about past paradigm shifts seems like the most scalable way to intuitively understand what deep, creative, principled research looks like.
@1a3orn maybe I will write this up into a proper reading group
Re feyerabend:
@agathomai Hmm, yeah, I was vacillating on that because there are good ideas in there but he’s a bit too postmodernist/destructive, he doesn’t have a positive account of what science is.
However, he is right at the top of the lending library I made for MATS:
@RichardMCNgo why Garrabrant induction as a model of scientific epistemology in particular? naively would consider {Against Method, Proofs and Refutations} to be the introduction
@RichardMCNgo Mechinterp (eg Circuits) and AI consciousness also seem important
@RichardMCNgo case studies of the idea of alignment and of mech interp seem appropriate too
@RichardMCNgo Popper seems like a good addition here: conjectures, refutations, fallibilism, criticism. @DavidDeutschOxf would likely agree, and The Beginning of Infinity is probably one of the best modern companions to that view.
@RichardMCNgo Randall Collins's Sociology of Philosophies, especially chapter 10 on the mathematical and scientific revolutions
@RichardMCNgo against method by feyerabend!