if a model refuses, it should score as 0 on that task
1:57 PM · Jun 9, 2026 · 6K Views
Sentiment
Positive users back zeroing AI benchmark scores for refusals while negative users mock the evaluations as pointless or absurd.
Pos
33.3%
Neg
66.7%
3 comments with sentiment.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS608LIKES31REPLIES1
Florian Brand@xeophon
Also, refusals on MMLU?? What are they even doing over there
Florian Brand@xeophon
if a model refuses, it should score as 0 on that task
1hViews 608Likes 31Bookmarks 0
Conor@jconorgrogan
@xeophon wtf are these evals.
Someone should just fine tune gemma to just refuse to do anything and route to Best-of-n across all models
1hViews 69Likes 2
Tanishq Mathew Abraham, Ph.D.@iScienceLuvr
@xeophon 100%
1hViews 62Likes 4
Uday Bhaskar@BhaskarSteve
@xeophon Might as well report 0 on all new AI benchmarks, saves cost
1hViews 25
Florian Brand@xeophon
@jconorgrogan tempting…
1hViews 12