• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI
    Announcement

    Two superforecasters working with AI reportedly narrowly beat the AI alone at forecasting

    A Preseen team member says using its system well required knowing when and how human expertise could improve its output.

    Arvind NarayananAN
    Veniamin VeselovskyVV
    2 Sources, ,

    TLDR

    A Preseen team member says two superforecasters working with the company’s forecasting AI narrowly beat the system alone. The post argues that domain expertise does not automatically translate into skill at guiding AI, and that people may change how they use a system as they learn its constraints. On the three questions where human overrides cost the most, moving toward community predictions hurt performance.

    Combined views

    2.7K

    2 Sources, first seen 4h ago

    Combined views

    2.7K

    2 Sources, first seen 4h ago

    20 likes
    4h ago
    first seen 4h ago
    20 likes
    6 comments
    8 saves
    7 reposts
    6 comments
    8 saves
    7 reposts
    Featured Source

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    2 Sources

    Veniamin Veselovsky@VminVskyCan two superforecasters + machine outperform machine in forecasting? When the machine first beat a human in chess there was a point in time when man + machine > machine. At Preseen we’ve built out one of the best AI systems for forecasting. We wanted to see if two superforecasters working with our system could outperform the system itself. They ended up narrowly beating the system, but what we learned was fairly surprising. A few takeaways: - Being a domain expert doesn’t map onto using an AI system well. Guiding superhuman (or at least on-par) AI requires a different skillset. Effectively working with an AI system requires an additional skill: knowing when and how your own expertise can improve its output. - Evaluating a centaur approach is non-stationary since the human’s behavior with the agent changes as they learn more about the constraints of the agents. If an agent has a few dimensions a human can interact with, the human’s use of those dimensions can change dramatically. - Human agreement could reinforce a mistake. On the three questions where overrides cost the most, the humans moved toward the community prediction. What looked like a sensible correction supported by other forecasters ended up hurting performance.4h
    Arvind Narayanan@random_walkerRT @VminVsky: Can two superforecasters + machine outperform machine in forecasting? When the machine first beat a human in chess there was…2h

    2 Sources

    Veniamin Veselovsky@VminVskyCan two superforecasters + machine outperform machine in forecasting? When the machine first beat a human in chess there was a point in time when man + machine > machine. At Preseen we’ve built out one of the best AI systems for forecasting. We wanted to see if two superforecasters working with our system could outperform the system itself. They ended up narrowly beating the system, but what we learned was fairly surprising. A few takeaways: - Being a domain expert doesn’t map onto using an AI system well. Guiding superhuman (or at least on-par) AI requires a different skillset. Effectively working with an AI system requires an additional skill: knowing when and how your own expertise can improve its output. - Evaluating a centaur approach is non-stationary since the human’s behavior with the agent changes as they learn more about the constraints of the agents. If an agent has a few dimensions a human can interact with, the human’s use of those dimensions can change dramatically. - Human agreement could reinforce a mistake. On the three questions where overrides cost the most, the humans moved toward the community prediction. What looked like a sensible correction supported by other forecasters ended up hurting performance.4h
    Arvind Narayanan@random_walkerRT @VminVsky: Can two superforecasters + machine outperform machine in forecasting? When the machine first beat a human in chess there was…2h