OPSD’s proposed role in fixing AI rule-following errors
A user suggests using exact reminders of existing instructions to address model failures, but sees no clear reason OPSD would outperform reinforcement learning.
TLDR
A user outlines a possible use for OPSD: identify violations of instructions already in a model’s context, insert a verbatim reminder just before the failure, and train only on the tokens immediately following it. They argue this avoids hint leakage and addresses rule-following problems in long contexts, but question whether it would beat reinforcement learning that penalizes the same failures. They also warn that training actions that conflict with preceding reasoning could raise concerns about the faithfulness of that reasoning. They see tool-schema failures as the clearest use case, while arguing that frequent tool-call mistakes likely point to bigger post-training problems.