@johnjhortonHot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagreeing cases, it's almost always that the LLM is right and the human(s) are wrong OR the humans also substantively disagree with each other
@ChenhaoTanLLM judges are underrated.
@minhnguyenRT @johnjhorton: Hot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagr…@johnjhortonHot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagreeing cases, it's almost always that the LLM is right and the human(s) are wrong OR the humans also substantively disagree with each other@ChenhaoTanLLM judges are underrated.@minhnguyenRT @johnjhorton: Hot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagr…@johnjhortonHot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagreeing cases, it's almost always that the LLM is right and the human(s) are wrong OR the humans also substantively disagree with each other@ChenhaoTanLLM judges are underrated.@minhnguyenRT @johnjhorton: Hot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagr…@johnjhortonHot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagreeing cases, it's almost always that the LLM is right and the human(s) are wrong OR the humans also substantively disagree with each other@ChenhaoTanLLM judges are underrated.@minhnguyenRT @johnjhorton: Hot take: From about GPT-5 onward, if humans and LLMs disagree in a data labeling task, when you look at the actual disagr…—
Not ranked yet
—
Not ranked yet
Comments