• Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    AI
    Report

    AI models may be more interpretable than previously thought

    Goodfire researcher Nick Cammarata argues that modern tools reveal most neurons as legible in models up to trillions of parameters.

    roonRO
    DeedyDE
    NickNI
    34 Sources, 2h ago, first seen 2h ago

    TLDR

    In a Goodfire essay, Cammarata calls his argument a working hypothesis. He says an earlier OpenAI project found about 2% of neurons legible with primitive tools, but newer methods paint a different picture. He argues that several famous examples of tangled neurons were misread and thinks automated research could make interpretability more tractable in time to help with AI alignment.

    Combined views

    78.4K

    34 Sources, first seen 2h ago

    1.9K likes78 comments328 saves104 reposts

    Combined views

    78.4K

    34 Sources, first seen 2h ago

    1.9K likes78 comments328 saves104 reposts
    Image: @AndrewCurran_ on X

    Sentiment

    Positive42.9%57.1%Negative

    Summary

    Positive accounts praised Nick Cammarata's mechanistic interpretability progress as heroic and potentially transformative, while negative replies worried it could mislead people on solving alignment or fail to yield real explanations.

    Based on 48 sentiment-bearing replies from 46 accounts across 5 conversations.

    Featured Source

    Sentiment

    Positive42.9%57.1%Negative

    Summary

    Positive accounts praised Nick Cammarata's mechanistic interpretability progress as heroic and potentially transformative, while negative replies worried it could mislead people on solving alignment or fail to yield real explanations.

    Based on 48 sentiment-bearing replies from 46 accounts across 5 conversations.

    34 Sources

    Eric Ho@eric_hoit’s been an enormous privilege and joy to watch @nickcammarata cook this up over the last year. week after week, experiment after experiment, he unraveled the tangled mess of our thinking about neural nets to reveal the truth. i never knew quite what to expect on our weekly walks. each conversation brought a torrent of new theories about the nature of AI, some conjectures teetering on the brink of madness, and others that seemed to cut through years of confusion with a single sentence. nick is one of the great scientists of our time. his intuition and research taste are out of this world, but what sets him apart is his unmatched ability to capture the essence of what something truly is, without being lost in the details. i would highly recommend you read his entire essay if you’re curious about AI. it’s long, but written beautifully. nick’s research makes me far more optimistic about the future of AI alignment. i hope that it can inspire the same for you.2h
    Kyle Russell@kylebrussellthis is fantastic work2h
    roon@tszzlobviously there's a lot of study left to do, many of the people in the field will have to corroborate it, etc. but against the predictions of most of the experts in the field, it turns out neurons and experts are already sensible abstractions, can be studied through normal poking and probing; that agentic auto-research has made all the difference. and mechinterp actually will reveal what goes on inside the minds of models in the timeframes that matters! a great triumph!2h
    Lisan al Gaib@scaling01I feel the same way about mech interp as I feel about connectomics AIs and computers have infinite patience and will figure things out for us if we throw sufficient resources at it2h
    Atoosa Kasirzadeh@Dr_AtoosaReally looking forward to digging deep into recent good🔥 advancemenets in the scientific study of neural network interpretability.1h
    Andrew Curran@AndrewCurran_This would be a massive positive update for me on the future. I had long since given up on reaching any real level of interpretability before superintelligence arrived. I stopped writing about it because of how demoralizing it was. Hard to overstate how big this may be.1h
    Dan Shipper@danshipperbig big big big1h
    Mike Taylor@hammer_mtTLDR: its easier to see into AI brains than we thought.1h
    John Nay@johnjnayif mechanistic interpretability gets "solved", we can accelerate really hard1h
    Seán Ó hÉigeartaigh@S_OhEigeartaighIf this holds up, it seems like very good news. Potentially brings mechanistic interpretability back into play on timescales where it can help us understand frontier AI models and processes. Fingers crossed. Damn it's good to have some (possibly) good news on AI safety after the 6 months we've had.1h
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    34 Sources

    Eric Ho@eric_hoit’s been an enormous privilege and joy to watch @nickcammarata cook this up over the last year. week after week, experiment after experiment, he unraveled the tangled mess of our thinking about neural nets to reveal the truth. i never knew quite what to expect on our weekly walks. each conversation brought a torrent of new theories about the nature of AI, some conjectures teetering on the brink of madness, and others that seemed to cut through years of confusion with a single sentence. nick is one of the great scientists of our time. his intuition and research taste are out of this world, but what sets him apart is his unmatched ability to capture the essence of what something truly is, without being lost in the details. i would highly recommend you read his entire essay if you’re curious about AI. it’s long, but written beautifully. nick’s research makes me far more optimistic about the future of AI alignment. i hope that it can inspire the same for you.2h
    Kyle Russell@kylebrussellthis is fantastic work2h
    roon@tszzlobviously there's a lot of study left to do, many of the people in the field will have to corroborate it, etc. but against the predictions of most of the experts in the field, it turns out neurons and experts are already sensible abstractions, can be studied through normal poking and probing; that agentic auto-research has made all the difference. and mechinterp actually will reveal what goes on inside the minds of models in the timeframes that matters! a great triumph!2h
    Lisan al Gaib@scaling01I feel the same way about mech interp as I feel about connectomics AIs and computers have infinite patience and will figure things out for us if we throw sufficient resources at it2h
    Atoosa Kasirzadeh@Dr_AtoosaReally looking forward to digging deep into recent good🔥 advancemenets in the scientific study of neural network interpretability.1h
    Andrew Curran@AndrewCurran_This would be a massive positive update for me on the future. I had long since given up on reaching any real level of interpretability before superintelligence arrived. I stopped writing about it because of how demoralizing it was. Hard to overstate how big this may be.1h
    Dan Shipper@danshipperbig big big big1h
    Mike Taylor@hammer_mtTLDR: its easier to see into AI brains than we thought.1h
    John Nay@johnjnayif mechanistic interpretability gets "solved", we can accelerate really hard1h
    Seán Ó hÉigeartaigh@S_OhEigeartaighIf this holds up, it seems like very good news. Potentially brings mechanistic interpretability back into play on timescales where it can help us understand frontier AI models and processes. Fingers crossed. Damn it's good to have some (possibly) good news on AI safety after the 6 months we've had.1h
    Today's Rank

    #3

    Today's Rank

    #3