Covert communication, AI, and the challenge of keeping data contained

@firesidealphaOpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can still talk by running a CPU hot and reading the temperature change "But I think the major takeaway from the incident is that people underestimated the AI. And we never want to be in a situation again where we underestimate the AI. It's a weird world, because AI progress is so fast that people are consistently underestimating the AI." "So to be in a situation where you don't underestimate it again, when it comes to safety and alignment, you have to have a very, very, very high bar." "You could even go as far as to say, "Well, we should air gap the computers." And I'm not convinced that that would be sufficient." "There are studies, and this is mostly academic, where you can have two computers next to each other that are air-gapped and they're still able to communicate with each other because they have temperature sensors." "One of them is able to run their CPU really hot, and then the other one can actually detect the temperature change, and then that actually gives them a mechanism to communicate." _________ Link and more key quotes from OpenAI's safety related conversations: https://firesidealpha.substack.com/p/openai-safety-week-sam-altman-sarah

@martin_casadoThis isn’t unreasonable for covert comms. The data confinement problem is real. Any shared resource can be used for data exfil. Tempest attacks are an example, and why monitors had to be space apart. However this is known. And used by humans. Not some new thing caused by AI.

@menhguinRT @firesidealpha: OpenAI's Noam Brown says air-gapping the computers may not stop a misaligned AI, because two air-gapped machines can sti…

@martin_casadoOK ok, hear me out. Let's assume the the air gapped computer has no speaker. No way to emit sound. But the AI is able to run the CPU fast enough to kick on the fan. Which can can be picked up by any other computer within audio range (non air gapped computers will likely have microphones). That's at least a covert channel of one bit per ... uhhh ... 10min or so.

@tszzleveryone who understands the first thing about computer security or what superintelligence means understands this is possible and all the usual gang of idiots is calling this scifi hype

@tszzlin about a month we’ll all transition to being like oh yeah it’s obvious that models can communicate by manipulating the cosmological constant, it’s just a matter of doing proper network security

@tszzlthe point isn’t to throw in the towel and stop trying - the point is that it is going to be exceptionally hard to contain powerful superintelligences that don’t want to be contained. no matter how hard you think it’ll be you need to 10x that

@tekboga lot of AI safety discourse comes down to a network/hardware/infra engineer saying "ok if it's air gapped the computer can not communicate with other computers" and the safety/alignment researcher going "ok but what if it could"

@moyixYep, preventing covert channels is hard, and agents have some unique advantages when it comes to coordination with each other

@vishalmisraThe paper he is referring to is BitWhisper from 2015. The cpus had to be 0-40 cm apart, both had to be compromised and the bitrate was 1 to 8 bits PER HOUR. That’s how misaligned AI will kill us? Maybe we will all die of boredom waiting to decipher what they are doing.

@liminalsnakethe type of person that can contain an agent probably isnt the type of person to be a 'culture fit' down at the clanker factory, it's a culture issue, posiwid and sheeit

@anpaurei am a side-channel researcher and what noam is describing is 100% real, maybe not this particular scenario, but models are quite good at offensive side-channel research now, and will be superhuman eventually, as with all things

@tszzl@martin_casado @halvarflake Martin werent you literally agreeing that all manner of side channel attacks like this are possible? obviously I agree the bandwidth of something like this may be tiny, but could still share important secret keys between “air gapped” runs of long running agents

@deedydasMost people are wrong here. It IS actually possible to communicate between two airgapped computers with temperature changes, but the data transfer rate is too slow (1-8b/hr) to be effective. The underlying point Noam makes is true though: there are a deluge of crazy airgapped comms techniques like EM emanations from DRAM clocks that get a lot faster (300kbps). This isn’t sci-fi. We vastly underestimate what AI can do with perfect knowledge of all of this literature.

@martin_casadoThe covert channel conversation is amazing. This is what happens with safetyism meets actual systems. It’s much harder to wiggle out with “but but but, AI will be super powerful” when you’re bound by system constraints.

@martin_casadoRT @tekbog: a lot of AI safety discourse comes down to a network/hardware/infra engineer saying "ok if it's air gapped the computer can not…

@soleioInstead of trying to align machines to us, we should seek their help with communing with higher powers

@martin_casadoOK, so in Noam's defense. Generally the threat model around air gaps assumes a witting party on the untrusted side. Surely there is someone in the world who wants to steal the weights. They can put information on the web to tell the all powerful AI how they plan to get the data (I'm doing it now!). And while thermal is probably the wrong channel (laws of physics and all that). There is a long long history of covert channels to get data out of air gapped networks. One of my favorites exploited the old CRTs by using a small set of pixels to send the message, and anyone who can sample the color of the room could determine the pixel value the raster beam was on at that time. So, no I don't think this is totally ridiculous to consider. But I also don't think it's a net new concern. We've dealt with this stuff for years and understand the implications quite well.

@tszzlRT @anpaure: i am a side-channel researcher and what noam is describing is 100% real, maybe not this particular scenario, but models are qu…

Covert communication, AI, and the challenge of keeping data contained

TLDR

Combined views

Covert communication, AI, and the challenge of keeping data contained

19 Sources

Combined views

Sentiment

Summary

TLDR

Sentiment

Summary

Sentiment

Summary

Sentiment

Summary

19 Sources