"The Hansonian in me says: the winning deal is a deal which, from the perspective of both sides' present-day beliefs & knowledge, both sides would accept, though for different reasons. If the crux is AI progress speed, then identify a set of pre-agreed triggers … [where] we…
I agree Plan A (AI 2040) isn't the right plan if AI is certainly a normal but important technology. But, the downside is modest and the full plan can wait for crazy capabilities.
Thus if you think Plan A is good given ASI by 2040, then it's good if ASI by 2040 is plausible.
I…
To me the strongest argument for delaying AI having very high humanitarian costs in a normal-tech world goes through the hypothesis that curing aging and globally providing extremely cheap universally available clean food, water and medicine may both be possible when and only…
@VitalikButerin In a "normal tech" world, I'd be very surprised if AI cures aging and massively increases material abundance. Maybe you're using "normal tech" very differently from me?
If AI is curing aging and 5x-ing GDP, I'm naively worried about takeover (by AIs or human…