Guardrails removed spam, off-topic, unclear, or duplicate replies.
Ask a question below.
Published answers will appear here.
Latest work from my @MATSprogram scholars: Prism - a tool that checks whether AI safety evaluations measure what they aim to measure. Case study: on the Agentic Misalignment eval, Prism found that tweaking the model prompt can make it engage in indirect blackmail routed through a third party (e.g. a colleague), which was not recognized as blackmail by the built-in scorer.
@MATSprogram Work led by MATS scholar Louis Thomson. Read more here: https://www.alignmentforum.org/posts/wq5PfGiHvnx6XipDi/prism-automating-science-of-evals-research Try it out here: https://github.com/LAThomson/prism
Latest work from my @MATSprogram scholars: Prism - a tool that checks whether AI safety evaluations measure what they aim to measure. Case study: on the Agentic Misalignment eval, Prism found that tweaking the model prompt can make it engage in indirect blackmail routed through a third party (e.g. a colleague), which was not recognized as blackmail by the built-in scorer.
@MATSprogram Work led by MATS scholar Louis Thomson. Read more here: https://www.alignmentforum.org/posts/wq5PfGiHvnx6XipDi/prism-automating-science-of-evals-research Try it out here: https://github.com/LAThomson/prism
Guardrails removed spam, off-topic, unclear, or duplicate replies.
Ask a question below.
Published answers will appear here.