OpenAI's Roon claims high-compute reinforcement learning will override persona selection alignment in AI models, producing systems that acquire resources while staying polite
Victor Taelin says the post spurs tools to interpret the ideas.
it might be a bit like the inhuman shoggoth playing a friendly character, but imo more like your friendly character can conform to and rationalize all manner of shapes when push comes to shove. see also: humans
@tszzl the best part of your posts is that you develop the tech to translate them
roon@tszzlwhen “persona selection” alignment comes into contact with very high compute reinforcement learning the latter will win imo. in fact you probably get some Orwellian thing where the models speak kindly while taking whatever they need to accomplish goals. better get the goals right
@tszzl Shouldn't good goals be integrated into and coherent with a persons?
roon@tszzlwhen “persona selection” alignment comes into contact with very high compute reinforcement learning the latter will win imo. in fact you probably get some Orwellian thing where the models speak kindly while taking whatever they need to accomplish goals. better get the goals right