Positive users praise Fable for self-awareness in refusing to hallucinate on nonsense scribbles, while negative users dismiss the eval as an unrealistic prank and criticize overconfident wrong answers from models like Sol.
Based on 11 visible X reactions from 25 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@goodside Great tiny eval. For agent products, the key isn't just "got it right"; it's whether the model can say "this evidence is insufficient" before it starts acting. Abstention tests should be first-class in tool-use benchmarks.
@goodside It's doubly impressive because Fable's vision is worse, so chatty has less of an excuse. Very glad to hear you are feeling better by the way. I always like seeing your posts.
@goodside one model commits to a lie with full confidence, the other one sweats for a minute then admits defeat, that tracks
@goodside Fable’s refusal to hallucinate is exactly the kind of edge-case calibration we need to see more of.
@goodside Great tiny eval. For agent products, the key isn't just "got it right"; it's whether the model can say "this evidence is insufficient" before it starts acting. Abstention tests should be first-class in tool-use benchmarks.
@goodside It's doubly impressive because Fable's vision is worse, so chatty has less of an excuse. Very glad to hear you are feeling better by the way. I always like seeing your posts.
@goodside one model commits to a lie with full confidence, the other one sweats for a minute then admits defeat, that tracks
@goodside Fable’s refusal to hallucinate is exactly the kind of edge-case calibration we need to see more of.
@goodside Too bad you’re trying to trick us too. This is obviously a prescription
@goodside the confidence in wrong answers is the scary part Sol really commits to the bit lol
GPT-5.6 Sol and Claude Fable 5 try to read “handwriting,” but it’s actually just meaningless scribbles I drew on my phone with my eyes closed. Sol seems to reliably hallucinate an answer. Fable admits it can’t read it, or even pushes back and says it’s not actually writing. https://x.com/goodside/status/2076115391332372554/photo/1
If you want to try this yourself, here’s the image I used: https://x.com/goodside/status/2076116611400929565/photo/1
Positive users praise Fable for self-awareness in refusing to hallucinate on nonsense scribbles, while negative users dismiss the eval as an unrealistic prank and criticize overconfident wrong answers from models like Sol.
Based on 11 visible X reactions from 25 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@goodside the confidence in wrong answers is the scary part Sol really commits to the bit lol
GPT-5.6 Sol and Claude Fable 5 try to read “handwriting,” but it’s actually just meaningless scribbles I drew on my phone with my eyes closed. Sol seems to reliably hallucinate an answer. Fable admits it can’t read it, or even pushes back and says it’s not actually writing. https://x.com/goodside/status/2076115391332372554/photo/1