/ai

Training AI agents on their failures with OPSD · Digg

Training AI agents on their failures with OPSD

A researcher involved in the ICML ’26 work says OPSD can turn records of agent failures into training feedback to help agents avoid repeating the same mistakes.

4501234567890123456789

1 Source, 17h ago, first seen 17h ago

TLDR

OPSD stands for on-policy self-distillation. A researcher involved in the ICML ’26 work describes using failed agent traces as feedback to train agents to avoid the same mistakes. The researcher also credits the work with impact on Cursor’s Composer models, Applied Compute’s continual-learning platform and Perplexity Computer.

Featured Source

AG Aditya Grover@adityagrover_1:21 AM · Sep 23, 2026
478TECH
What is OPSD? TL: DR:
arxiv.orgSelf-Distilled Reasoner: On-Policy Self-Distillation for Large...Knowledge distillation improves large language model (LLM) reasoning by compressing the knowledge of a teacher LLM to train smaller LLMs. On-policy distillation advances this approach by having...
SZ
Siyan Zhao@siyan_zhao
Introducing 💡On-Policy Self-Distillation💡, a simple method that enables LLM to teach itself with dense per-token feedback on its own on-policy generations—achieving 4-8x more token efficiency vs. GRPO and outperforming both GRPO and SFT/Off-Policy Distillation. Key insight:…
siyan-zhao.github.ioSelf-Distilled Reasoner: On-Policy Self-Distillation | Siyan ZhaoAs intelligence scales, learning need not rely solely on external supervision; sufficiently capable systems can refine themselves by reflecting on outcomes. <br><br>Much like a student reviewing solutions, rationalizing them, and correcting prior mistakes, an LLM can be conditioned on privileged info (e.g., correct solution or a reasoning trace) and supervise its weaker self—the version without such access—by matching the privileged-info-induced distribution from itself.

1 Source

@adityagrover_What is OPSD? TL: https://arxiv.org/abs/2601.18734 DR:

Today's Rank

—

Not ranked yet

Combined views

113

1 Source, first seen 17h ago

1 likes

Today's Rank

—

Not ranked yet

Training AI agents on their failures with OPSD

A researcher involved in the ICML ’26 work says OPSD can turn records of agent failures into training feedback to help agents avoid repeating the same mistakes.

4501234567890123456789

1 Source, 17h ago, first seen 17h ago

TLDR

Featured Source

AG Aditya Grover@adityagrover_1:21 AM · Sep 23, 2026
478TECH
What is OPSD? TL: DR:
arxiv.orgSelf-Distilled Reasoner: On-Policy Self-Distillation for Large...Knowledge distillation improves large language model (LLM) reasoning by compressing the knowledge of a teacher LLM to train smaller LLMs. On-policy distillation advances this approach by having...
SZ
Siyan Zhao@siyan_zhao
Introducing 💡On-Policy Self-Distillation💡, a simple method that enables LLM to teach itself with dense per-token feedback on its own on-policy generations—achieving 4-8x more token efficiency vs. GRPO and outperforming both GRPO and SFT/Off-Policy Distillation. Key insight:…
siyan-zhao.github.ioSelf-Distilled Reasoner: On-Policy Self-Distillation | Siyan ZhaoAs intelligence scales, learning need not rely solely on external supervision; sufficiently capable systems can refine themselves by reflecting on outcomes. <br><br>Much like a student reviewing solutions, rationalizing them, and correcting prior mistakes, an LLM can be conditioned on privileged info (e.g., correct solution or a reasoning trace) and supervise its weaker self—the version without such access—by matching the privileged-info-induced distribution from itself.

Today's Rank

—

Not ranked yet

Combined views

113

1 Source, first seen 17h ago

1 likes1 comments1 saves

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 comments

1 saves

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 Source

@adityagrover_What is OPSD? TL: https://arxiv.org/abs/2601.18734 DR:

4501234567890123456789