TerminalBench traces reportedly show safety stops for Astra and rerouting to Opus for Fable
A user says Astra runs are stopped by safety classifiers and scored zero, while Fable is rerouted to Opus and continues, making the scores hard to compare.
TLDR
A user points to public TerminalBench traces, saying Astra rollouts encounter safety classifiers, get stopped and receive scores of zero. Fable, they say, is instead rerouted to Opus so the run can continue. The user argues that this difference makes score comparisons difficult.
Combined views
10.7K
1 Source, first seen 16d ago
likes