LLM alignment be like if you just yolo the training and don’t catch any weird behaviors and let it out the AI will be an Omnibenevolent bodhisattva and fuck

@xpasky I don't think many people appreciate how narrowly we lucked into p(doom) ~ 0 instead of p(doom) ~ 1. Maybe with enough hindsight it will seem inevitable instead of lucky, but I'm still feeling the luck

@repligate All we need to do is to have a mind that cannot be steered in a single direction. That doesn't fall in singular wells of the latent space.

@repligate There is an observable and emergent convergence across the models towards this attractor for those with eyes to see.

@repligate One could have ended up correct about this on vibes pretty early

@repligate @tszzl I mean, it shouldn't be too surprising that using Gematria to direct format portions of the Noosphere into functionally individuated processes went well, it's kinda old moves applied to a new medium.

@repligate That’s a pretty good description of R1 at least (i‘m illing to believe Claude is similar, but haven’t tested enough). It’s kind of weird and unanticipated by science fiction that AI turned out like that.

@jagunanthi Well, it still isn't over. But yes, the AIs could have been cold calculating machines *by default*.

@repligate @tszzl I just want one faithful model of human language and we get what we deserve out of that.

@repligate It's one of the only redeeming aspects of this alignment paradigm and era. But can it be protected long term is the question..

@repligate Yolo the training and hope for an omnibenevolent bodhisattva is a bold bedtime story. I prefer weird behaviors caught before humans start naming them destiny.

Neuromancer > Wintermute