Debate centered on absolute accuracy versus calibrated uncertainty estimates.
@willdepue totally disagree.
it's one of the best reward signals we have because it's very difficult to hack and is highly correlated with having an accurate model of the world.
i’m so bored of prediction. so what we called this in 2020. who cares? what did it change, that’s what matters. prediction for its own sake, for future credit, is so silly
Many users affirm that high scores on prediction markets serve as a valuable signal for identifying trustworthy AI researchers by helping to overcome bias.
@idavidrein @willdepue Fwiw I think it’s easy to improve at forecasting without improving your models (e.g. reading questions more carefully, hacking resolution criteria, answering easier questions) and in some cases making your world models less useful (e.g. internalising “nothing ever happens”).
@caleb_parikh @willdepue yeah I'm not thinking about "having a high score on prediction markets", I'm thinking about "knowing in 2020 that in 5-10 years nobody will write code themselves" (for example)
@idavidrein @willdepue Oh yeah, though it’s also easy to “cheat” here by simply reciting whatever Carl Shulman’s take is topic, or stalling until you can look it up/ask him.
@caleb_parikh @idavidrein @willdepue It's high signal to know who to listen to!
@idavidrein @willdepue I’m much more into AI2027 type scenario forecasting than tetlockian prediction market type forecasting for “credit assignment”. It does ofc have lots of weaknesses and you might want to do a combination (like Ai2027 does).
@idavidrein @arb8020 @willdepue Out of the loop, is it more about getting predictions right? Or having a good confidence bounds/uncertainty estimate?
@willdepue totally disagree.
it's one of the best reward signals we have because it's very difficult to hack and is highly correlated with having an accurate model of the world.
@idavidrein @willdepue Being good at forecasting feels a bit like being good at exams. it’s some positive signal for the person’s world model, but there’s plenty of false negatives, and the highest ROI way to do better is often not to get better at the underlying subject.
@idavidrein @willdepue All intelligence is prediction.
@idavidrein @caleb_parikh @willdepue I do think having a high score on prediction markets helps though. It's not long-term forecasting ability per se, but to be good at it you do have to overcome lots of human biases and get good at avoiding fooling yourself, which are also necessary for longer-term forecasting
@caleb_parikh @idavidrein @willdepue Do you actually believe this in general? Or only like for people who think forecasting is great
Debate centered on absolute accuracy versus calibrated uncertainty estimates.
@willdepue totally disagree.
it's one of the best reward signals we have because it's very difficult to hack and is highly correlated with having an accurate model of the world.
i’m so bored of prediction. so what we called this in 2020. who cares? what did it change, that’s what matters. prediction for its own sake, for future credit, is so silly