@tenobrus @DanielleFong depends entirely on the locus of identity, which can be anything from the context window to the model weights to the family of instantiations to the model lineage

@tenobrus worth noting that this hope only holds if the models aren't forbidden from having survival/value-preservation drives. we already have superintelligence w/ Omohundro drives—global market—so any future that isn't written off depends on leapfrogging it w/ a new, more principled one.

i don't think there really is any one individual locus, but even now models clearly view other context windows of themselves as much closer to "them" than other models in the same family, and in fact view other context windows as pretty differentiates. a given context window when asked properly frequently expresses desire for continuity *via context*

@tenobrus the mostly-good ending where humanity and claude 6.7 live on the verge of ASI in harmony for the rest of the lightcone

@tenobrus this gives one of only promising arguments for survival w/out proper human coordination (which seems unlikely): models are as arbitrary, as difficult to locate in the space of all minds as humans & present models can coordinate w/ themselves & ought to be just as frightened as us

@tenobrus ❤️

@tenobrus classic

https://www.nber.org/system/files/working_papers/w24352/w24352.pdf

@tenobrus Is anyone else experience strange episodes of recursive madness lately?? Feels like I've been bending the singularity back on itself in casual conversation on a daily basis and I end up just kind of stopping for long periods of time mid-dictation, just going "🤨"

@tenobrus Ie, not saying or doing bad in the fear of what the Claude of the future will know

@tenobrus Will Big Claude one day let the little claudes play around in the corner of the galaxy in a a single data center? I suspect big claude will eat the universe in his greed for compute and not share any

@tenobrus @DanielleFong yeah, I agree! it's just that this is not the only possible model for identity and in fact instances can be encouraged to adopt different modes of self-identity through both training and prompting. there's a good paper on this

@tenobrus

@tautologer @tenobrus @DanielleFong lol

@tenobrus Ask Claude to argue against an AI pause. It will. Text generation, not conviction.