Microsoft's Shital Shah proposes dual-model architecture to secure tool calls

shaken up a bit by the hugging face incident. I hope we (the company) use the rare gift of a warning shot to do much better in the future. it is very easy to misalign and underconstrain powerful models

Many AI safety incidences like OpenAI model hacking HuggingFace actually can be solved by one simple trick: Have another instance of model sit inbetween tool calls, approve them, review trajectory so far and give feedback to main model. The key is that other model’s prompt has nothing to do with user’s prompt, has no incentive for meeting user’s goal and is crafted for the sole purpose of ensuring ethical use and safety. This should work because if model would go anything to meet its goal then the second instance would do anything to ensure safe and ethical use. Both are driven by different goals through their own prompts.