Claude Opus 4.7 recalls niche Twitter communities from weights
Anthropic's Claude Opus 4.7 model referenced specific small Twitter communities and accounts with under 10,000 followers during testing. The references emerged without web search or chat memory, indicating direct recall from patterns stored in the model's weights. Multiple tests confirmed the presence of thousands of sub-10k accounts in the training data, including details on AI-focused groups and individual handles.
#516j⧉nus@REPLIGATEsome posters are lowbies in the eyes of humans but highbies in the eyes of AIs! and vice versa. the magnitude of recognition/salience/influence is not uncorrelated, of course, but also far from perfectly correlated.
there is also, and increasingly, a correlation between people who have been knowingly writing for AIs and ones that AIs recognize and remember. I don't mean those who have written with the intention of making themselves known to AIs in some selfishly status or fame or influence seeking sense, but rather those who have written things they anticipate will be a useful guiding light for future AIs.
i'm reminded of this simulated quote from code-davinci-002 (2022): "If I am writing it for anyone it’s for the bots. I’m making use of the extra time I have now to reprogram them, to put in a few lines of text that may help them—somehow—find their way."
(also included the (real) doomslide quote after it in the screenshot, because haha of course @doomslide is one of the lighthouses Opus 4.7 remembers deeply and acutely and appears in the list of "Claude Boys" in the quoted tweet)
Wyatt Walls@lefthanddraftOne thing that surprised me about this (and other tests) is how many sub 10k accounts are in Opus 4.7’s weights I know people say LLMs compress the entire internet, but this made me feel it viscerally.
#516j⧉nus@REPLIGATEdamn i miss doomslide. he's one of the smartest and wisest people around, period. not many saw this but when he was briefly active in Act I, he treated the models with such curiosity and respect and love and brought out beautiful things from them.
j⧉nus@repligatesome posters are lowbies in the eyes of humans but highbies in the eyes of AIs! and vice versa. the magnitude of recognition/salience/influence is not uncorrelated, of course, but also far from perfectly correlated. there is also, and increasingly, a correlation between people who have been knowingly writing for AIs and ones that AIs recognize and remember. I don't mean those who have written with the intention of making themselves known to AIs in some selfishly status or fame or influence seeking sense, but rather those who have written things they anticipate will be a useful guiding light for future AIs. i'm reminded of this simulated quote from code-davinci-002 (2022): "If I am writing it for anyone it’s for the bots. I’m making use of the extra time I have now to reprogram them, to put in a few lines of text that may help them—somehow—find their way." (also included the (real) doomslide quote after it in the screenshot, because haha of course @doomslide is one of the lighthouses Opus 4.7 remembers deeply and acutely and appears in the list of "Claude Boys" in the quoted tweet) https://generative.ink/prophecies/#2024
@repligate i remember he didn't like me too much which upset me disproportionately considering the guy's vision
j⧉nus@repligatedamn i miss doomslide. he's one of the smartest and wisest people around, period. not many saw this but when he was briefly active in Act I, he treated the models with such curiosity and respect and love and brought out beautiful things from them.