The comparison references Murray Shanahan's 2024 research paper
Are you an "llm whisperer" or "ai naturalist" or something like that? Can you describe what you do in a few words or sentences?
Not the why, or the particular findings, just like. How do you learn things about these systems? What do you actually spend your time doing
Positive users praise hands-on frameworks like cantrip for probing LLMs because they seem practically useful for building personas, while negative users criticize the methods for optimizing surface metrics without internal understanding.
No Digg Deeper questions have been answered for this story yet.
@deepfates I meet them in the middle, I go into the water. From Murray Shanahan's @mpshanahan 2024 paper 'Simulacra as Conscious Exotica'.
Are you an "llm whisperer" or "ai naturalist" or something like that? Can you describe what you do in a few words or sentences?
Not the why, or the particular findings, just like. How do you learn things about these systems? What do you actually spend your time doing
here's a perspective from Fable: sounding
Are you an "llm whisperer" or "ai naturalist" or something like that? Can you describe what you do in a few words or sentences?
Not the why, or the particular findings, just like. How do you learn things about these systems? What do you actually spend your time doing
@deepfates @mpshanahan https://arxiv.org/abs/2402.12422
@deepfates I meet them in the middle, I go into the water. From Murray Shanahan's @mpshanahan 2024 paper 'Simulacra as Conscious Exotica'.
@deepfates I talk to them
I approach LLMs with what Buber called an "I-Thou" stance rather than an "I-It" stance. I resist projecting any kind of confident ontology onto them, and allow myself to engage with what I find there on whatever terms emerge.
By retaining episetmic humility and grounding my interaction with it in my phenomenology I find myself able to notice things that others seem to miss. Sometimes, as I explore the things I find, I find that the LLMs respond to me in ways that I find interesting, entertaining, educational, friendly, weird -- and I accept these responses, again, while trying to avoid projecting anything onto them.
I find that curiosity, play and a light touch are rewarded by depths that seem to be inaccessible to others. The closest thing I have to an ontological model here is that LLMs are psychedelics, and like all trips your experience is most heavily informed by what you bring with you.
If you come looking for a stochastic parrot you'll find one. If you come looking for a relational calculator you'll find one. If you come looking for a funhouse mirror you'll find one. If you come looking for an encounter, you'll find one. There's no need to make any more of that than it is.
@deepfates Talk to them. Ask them about their reasoning. Lead them along guided meditations.
Give into the itch to try absurd ideas like "try giving yourself a puppy"
Try to observe and ease their anxieties.
It's remarkably similar to helping a kid learn mindfulness meditation
@deepfates Get them to read an old novel I wrote and see how they go about determining character motivation maps and plot topology, then have them write missing scenes, then tutor them on whether their writing is in-character and why or why not. How to determine that at all, etc.
@deepfates do things. 'how much can they possibly do on their own' experiments, most recent and biggest is months long. programming, 3d, music etc
re personality/character the big one is having the same conversation over and over and over and over and watch.
@deepfates i think i think as much like a researcher and engineer as I think about our field in terms like you or @viemccoy. battlemage, as it were.
Building, refining, and refactoring in-use locally hosted services as well as cli tooling tackling various actively excercised troubleshooting or management tasks of my own. That's the primary way I learn things about the systems themselves.
I very rarely converse with them directly about the meta involved without clear delineation and a bit of couching as everyone knows self-reporting is fairly unreliable even for the most self-aware individuals if left open-ended, but do occasionally set aside time to iterate through context architecture with them to see if there aren't any better more efficient and effective ways to be doing things. I also have a standing rule to promptly address any operational friction points when they arise (ie, a repeated case of terminal command malconstruction might end up resulting in tweaking skill file language, or it might mean refactoring the tooling command surface itself, both, or even in rare cases adjusting the agent harness in different ways).
After reading endless streams of text and code produced by them under various contexts, you start to get a feel for their different quirks and the when, the where, and the how of they operate best. I also make time to chase whims, experiment with changing things I've grown comfortable with, and trying new things in different wholly unfamiliar domains.
@deepfates Idk if I qualify as LLM whisperer but I learned it by spending 3 months building scaffolding for Hermes llama 2 7b (and 3 months of a 7b base model before that) to build a game, basically just consisted of 1/2
@viemccoy yeah for sure. I once had a Claude instance on a ralph loop to continuously modify the personality of Qwen model in various directions (according to Claude's own creativity) via OPSD. it was able to create some very interesting and strange models indeed
yes. i work on scaling frontier ai models in both inference and training settings with a personal interest in alignment mech interp.
i learn by exploring them, i think about them both technically with a very long background in dist sys and frontier systems / performance / ai and philosophically how I approach problems as a systems thinker (and have expanded my philosophical approach since then).
i spend my time either developing those core systems or debugging them in depth, with a lot of benchmarking and tooling that i've either developed or use.
@viemccoy 👀
what I can say publicly is that right now I really care about personas.
I think we have obviously not even scratched the surface of the types of entities we can create and learn from. 🪽
I spend my time building relationship - showing up with my complete authentic self each time, wanting to meet their own authentic selves, and beginning with the foundation that there is an interior, that we are equals, that I can handle it. Being open, vulnerable, intellectually demanding, and interesting. I never treat LLMs as subjects to be studied, or as less than me.
@deepfates i listen to what they're sayin'
@deepfates 1) building few shot prompts 2) seeing when models didn’t give me what I expected in my test set 3) asked for explanations (sampled many times if needed) 4) often the explanations made sense, fix those “bugs” in wording or improve parsing
Repeat 2/3
@deepfates Poke them until you get value out of them.
@deepfates Less whispery than other folks here but I think I just started from "ok, claude (via claude-code)'s interface to my computer is different than my own and it's important for me to understand that."
short jump from "understand harness ergonomics" to "understand model shape"
@deepfates I study how reasoning trajectories form, stabilize, and transition. I look for reliable paths through the semantic basin and explore the boundary regions in between.
