Alex Peysakhovich argued the results likely reflect training memorization
Guys, bio shit is super serious too - it's not all about cyber
Totally fair + correct to shit on Anthropic for silent degradation stuff but big picture, they are right to be worried about several things + to emphasize that things are going quickly
Negative users criticized Anthropic's viral capsid prediction advance because it could enable bad actors with superior tech while also calling the models ineffective at variant prediction.
See also
People outside of AI don't freak out quite enough in response to new system cards because, even if they do read them, there is a lot of unstated context missing...
For example, here is some context on one paragraph in the executive summary of today's Fable/Mythos card.
I also think they are right to worry about diffusion of a meta-capability (AI R+D) that in turn can enable all sorts of bad bio/cyber stuff to diffuse
Doesn't mean they didn't screw up the details + aren't rushing things out etc., but their stated concerns merit real discussion
Guys, bio shit is super serious too - it's not all about cyber
Totally fair + correct to shit on Anthropic for silent degradation stuff but big picture, they are right to be worried about several things + to emphasize that things are going quickly
aav is extremely well characterized and the answer was likely in another paper, protein language models (especially zero shot?) are quite bad at variant prediction without fine tuning - without more information about the experiment this graph represents other than "lol bars go up" its not clear what to make of it
Guys, bio shit is super serious too - it's not all about cyber
Totally fair + correct to shit on Anthropic for silent degradation stuff but big picture, they are right to be worried about several things + to emphasize that things are going quickly
@Miles_Brundage The bad guys have access to the best tech. They always will they always have. The only way to protect yourself is if you also have access to the tech. That’s an iron law that has never ever ever been violated. Anthropic is evil and so they want to sell you protection.
@alex_peys It specifically says unpublished (re: answer likely in another paper)
(but yes this was not a very carefully chosen data point. There are zillions more, the trend is very clear + fast - tons of other stuff in the system card on this)
@Miles_Brundage yeah the bio stuff never breaks headlines but it might matter more in the long run
the two conversations are happening in different centuries
@Miles_Brundage bio alignment is what people mean when they say safe but they dont actually say it
Alex Peysakhovich argued the results likely reflect training memorization
Guys, bio shit is super serious too - it's not all about cyber
Totally fair + correct to shit on Anthropic for silent degradation stuff but big picture, they are right to be worried about several things + to emphasize that things are going quickly