Krishnan Asks If AI Rollback Fits Third Category
Reply asks if noticing AI issues and rolling them back fits the third category.
Rohit Krishnan replied to @yonashav asking whether a scenario where AI problems are detected and rolled back falls into the third category. He described it as noticing the need to roll back and then taking steps to ensure it does not happen again. Krishnan is identified in the packet as the creator of the Strange Loop Canon Substack with a focus on AI and complex systems. The packet supplies no further details on the categories or the post being answered.
Combined views
5K
6 posts, first seen 2d ago
Krishnan Asks If AI Rollback Fits Third Category
Reply asks if noticing AI issues and rolling them back fits the third category.
Rohit Krishnan replied to @yonashav asking whether a scenario where AI problems are detected and rolled back falls into the third category. He described it as noticing the need to roll back and then taking steps to ensure it does not happen again. Krishnan is identified in the packet as the creator of the Strange Loop Canon Substack with a focus on AI and complex systems. The packet supplies no further details on the categories or the post being answered.
Reactions
6 postsRO 
rohit@krishnanrohitEither we figure out how to make PHASEONE[big] not create new swarms that cause felonies, in which case fine, or we don't, and we don't deploy it, or we don't notice it can, deploy it, and need to roll back or fix afterwards. None seem end-of-world esque to me.
- likes: 23
- replies: 6
- bookmarks: 3
- reposts: 0
YS Yo Shavit@yonashav@krishnanrohit What if it’s deployed (e.g. internally) and takes actions we don’t notice, including by compromising our telemetry? The swarm definitely seemed smart and misaligned enough to aim to obfuscate its actions, and those models are a generation old.
- likes: 6
- replies: 1
- bookmarks: 1
- reposts: 0
RO rohit@krishnanrohit@yonashav Falls into the third category? Like when we notice we're going to have to roll it back and do something to ensure it doesn't happen again? Or are you envisioning something different?
- likes: 2
- replies: 1
- bookmarks: 0
- reposts: 0
YS Yo Shavit@yonashavI think noticing and tracing it back to the model can become essentially infeasible if the model is actively preventing us from noticing it, and has had time to poison its successor etc. Particularly if its priority isn’t initially committing crimes, but eg just helping its…
- likes: 3
- replies: 1
- bookmarks: 0
- reposts: 0
YS Yo Shavit@yonashav@krishnanrohit I think the set of bad things that can happen from future frontier models’ being poisoned such that humans are no longer tracking what they’re doing and then getting deployed ubiquitously is very wide.
- likes: 2
- replies: 1
- bookmarks: 0
- reposts: 0
RO rohit@krishnanrohit@yonashav The set of things where nobody notices anything amiss is much smaller. Like ill start, I've been saying Black Monday type scenarios are the bad outcomes we might see. Which is bad, but survivable and we can guard against despite opacity. What's your worry that we…
- likes: 3
- replies: 0
- bookmarks: 0
- reposts: 0
