This example is from Rohin Shah, not Caleb, but I nonetheless think it's notable. My understanding is that 3 years ago people thought something like "some data is good to include and up sample, some data (e.g. evals) is good to filter" and current views are similar though more specific (and depend on stuff like whether it's possible to add data back at the end): https://www.lesswrong.com/posts/dEiBJDtSmbC8dChwe/what-training-data-should-developers-filter-to-reduce-risk-1 I'm not saying people's views haven't shifted, many shifts have happened, but at least on my own views they tend to be moderate updates or making some fuzzy area more precise. More generally I think a key issue is that I expect that by the time various problems are legible, it's either too late or requires drastic intervention. And most types of generically improving defensive capacity don't seem to help that much. Though they help some and great institutions that actually have power (or that people defer to) could be massive. I think we can probably get some significant risk reduction by just doing measures that have a pretty solid and legible case at the time and don't require much foresight. Like measures that are justified by the current or obvious near future level of capability and issues. So people should pursue this and this is important. (I'm excited about some of the stuff that e.g. metr is planning to do here.) But I think this maybe only cuts risk by around 1/4 in the ideal case (e.g. perhaps optimistically reducing AI takeover risk from 45% to 35%). And I think it's very doable to make reasonable predictions in advance! So taking advantage of this seems great.

I roughly agree with this essay about the importance of making decisions on the fly, but think it understates how much foresight we can (and have!) had. The main example it uses ("consensus was it's great to include alignment data and now it's the opposite") is also false! https://twitter.com/calebwatney/status/2075260079666843847