• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Roon Says AI Labs Should Fund Alignment NGOs

    Roon argues labs offload alignment work onto NGOs and should pay billions in return.

    RO
    MH
    SÓ
    4 Sources, 46d ago, first seen 46d ago

    TLDR

    Pseudonymous OpenAI staff member @tszzl posted that frontier model companies freeload off alignment NGOs including METR, Redwood, Apollo, and Goodfire. He stated the labs offload sizeable alignment externalities onto these groups and should inject billions into organizations they support. In a visible reply, researcher Seán Ó hÉigeartaigh added that UK taxpayers also fund such work through the AI Safety Institute. The posts discuss responsibility for covering alignment costs tied to model training.

    Combined views

    70.3K

    4 Sources, first seen 46d ago

    Combined views

    70.3K

    4 Sources, first seen 46d ago

    1.1K likes
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    1.1K likes
    62 comments
    200 saves
    111 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    62 comments
    200 saves
    111 reposts

    4 Sources

    @tszzlthe model training cos freeload tremendously off of NGOs like METR, Redwood, Apollo, and smaller alignment cos like Goodfire. they offload sizeable share of their alignment externalities onto them. they should be injecting billions into organizations they believe in in some of these cases that’s easy, but in other cases like METR - understandably works very hard to preserve financial independence from the labs. there is a philanthropic market for a third party (I like Coefficient Giving but unfortunately too close to Anthropic) that can credibly be independent of the labs, still accept donations from lab employees, or stretch goal, the labs directly and distribute it to promising organizations and people studying alignment and control some philanthropy people will groan but OpenAI foundation and similar could offer huge prizes for technical alignment problems or research with verifiable results in other words the whole ecosystem deserves an injection of some of the vast riches of the labs if this industry is going to realistically self regulate
    @S_OhEigeartaigh@tszzl Don't forget the uk taxpayer through AISI
    @MariusHobbhahnAgree. With the current trajectory, it would probably be in labs' direct egoistic interest to do this. Looks like the alignment problems are already directly harming the usability of their models. So spending comparatively small amounts of money for them seems totally rational
    @lukeprogRT @tszzl: the model training cos freeload tremendously off of NGOs like METR, Redwood, Apollo, and smaller alignment cos like Goodfire. th…

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    4 Sources

    @tszzlthe model training cos freeload tremendously off of NGOs like METR, Redwood, Apollo, and smaller alignment cos like Goodfire. they offload sizeable share of their alignment externalities onto them. they should be injecting billions into organizations they believe in in some of these cases that’s easy, but in other cases like METR - understandably works very hard to preserve financial independence from the labs. there is a philanthropic market for a third party (I like Coefficient Giving but unfortunately too close to Anthropic) that can credibly be independent of the labs, still accept donations from lab employees, or stretch goal, the labs directly and distribute it to promising organizations and people studying alignment and control some philanthropy people will groan but OpenAI foundation and similar could offer huge prizes for technical alignment problems or research with verifiable results in other words the whole ecosystem deserves an injection of some of the vast riches of the labs if this industry is going to realistically self regulate
    @S_OhEigeartaigh@tszzl Don't forget the uk taxpayer through AISI
    @MariusHobbhahnAgree. With the current trajectory, it would probably be in labs' direct egoistic interest to do this. Looks like the alignment problems are already directly harming the usability of their models. So spending comparatively small amounts of money for them seems totally rational
    @lukeprogRT @tszzl: the model training cos freeload tremendously off of NGOs like METR, Redwood, Apollo, and smaller alignment cos like Goodfire. th…