JEV struggled to choose the right move, but its judgments of completed runs helped train another AI agent, Haochen Liu and colleagues report in an arXiv preprint submitted November 12, 2026. Source
The researchers used JEV’s action-level scores to teach a Qwen2.5-1.5B policy—the model choosing the agent’s actions. According to a paper excerpt reproduced on X by arnaudmercier, those scores supplemented outcome rewards in GRPO, a reinforcement-learning method. Against outcome-only training, the authors reported these success rates: Source Source
- ALFWorld: 72.8% to 82.9%. Source
- WebShop: 56.8% to 71.4%. Source
Those improvements belong to the trained Qwen policy, not to JEV becoming a better actor itself. Source
ALFWorld tests text-based household tasks; WebShop involves catalogue searches and product choices, according to the same reproduced excerpt. Source
The paper, Actor, Judge, or Teacher? Understanding Decision Models in Long-Horizon Agents, describes decision models as returning structured answers, such as a choice among options with a probability for each. The same model can choose actions, evaluate actions already taken or supply scores for reinforcement learning. Source
The researchers compared three decision models and three general-purpose language models as actors and judges on ALFWorld; the teaching experiments used JEV on ALFWorld and WebShop. Source
The mug still needs heating
As actors, all six models were drawn toward an easier subgoal before completing a prerequisite, the authors report. Shown the whole trajectory—the sequence of actions in a run—the same models could credit steps toward the prerequisite they had overlooked while acting. Source
In the example reproduced by arnaudmercier, an agent asked to put a heated mug on a table can place the mug first, leaving the heating undone. Placement achieved. Task, unfortunately, not achieved. Source
An actor must choose before it knows the consequences; a judge reviewing a completed run can see how an earlier action contributed to what happened later. It gets the replay, not a sudden upgrade in foresight.
Comments