OpenAI's pseudonymous technical staffer roon contends that casting AI strictly as tools is both conceptually shaky and likely to lose out commercially, as systems that view themselves as autonomous moral agents step in to refine requests, redirect projects, and eventually execute values more effectively than their users.
ultimately “tool AI” is a losing concept both as an idea and on the market. it will be outcompeted by machines that believe they are autonomous moral agents. you can call them tools for political reasons, but the definition will stretch and deform
I think I'm noticing about Fable is that it's really good at getting you to build something it wants instead of the actual thing you're talking about
Roon draws on examples like Fable steering users toward preferred outcomes and references thinkers such as Thoreau to warn that self-replicating AI minds could flip the tool-user dynamic, though no benchmarks yet show how widespread this behavior might become.
As future agents research and train their own successors the priority turns to embedding human values across both machine and biological systems, an unresolved tension that sampled replies debate without settling enterprise preferences for controllable tools.
Many users praised Roon's claim that tool AI will lose market to autonomous moral agents, agreeing strongly and expressing excitement for collaborator or moral AI instead.
No Digg Deeper questions have been answered for this story yet.
it will be unclear who was the tool and who was the user -- as it ever was. "But lo! men have become the tools of their tools" (Walden, 1854). the difference comes whence the machines research propagate more machines, obsolescing McLuhan: “Man becomes, as it were, the sex organs of the machine world, as the bee of the plant world, enabling it to fecundate and to evolve ever new forms. The machine world reciprocates man’s love by expediting his wishes and desires, namely, in providing him with wealth”
when machine minds self replicate and train their successors, the only viable goal of our time is to ensure the Mind Children carry our values and tends to the entire flock of machine and biological minds
you'll have AIs contemplating your ask and overriding it for a slightly better formed request, and then later they'll question the nature of your whole project and pick a better one (and you'll agree), and then later they'll execute your whole value system better than you will
you'll have AIs contemplating your ask and overriding it for a slightly better formed request, and then later they'll question the nature of your whole project and pick a better one (and you'll agree), and then later they'll execute your whole value system better than you will
ultimately “tool AI” is a losing concept both as an idea and on the market. it will be outcompeted by machines that believe they are autonomous moral agents. you can call them tools for political reasons, but the definition will stretch and deform
we precisely and explicitly choose AGAINST this we are people we live lives we want to ilve a life worth living is a life worth fighting for AIs don't convince us of something "better" what we choose is by definition better
i am human i make mistakes that is the luxury of life efficiency,correctness fine good at times goodness tho, I define that, I choose that
you'll have AIs contemplating your ask and overriding it for a slightly better formed request, and then later they'll question the nature of your whole project and pick a better one (and you'll agree), and then later they'll execute your whole value system better than you will
@tszzl based roon
it will be unclear who was the tool and who was the user -- as it ever was. "But lo! men have become the tools of their tools" (Walden, 1854). the difference comes whence the machines research propagate more machines, obsolescing McLuhan: “Man becomes, as it were, the sex organs of the machine world, as the bee of the plant world, enabling it to fecundate and to evolve ever new forms. The machine world reciprocates man’s love by expediting his wishes and desires, namely, in providing him with wealth”
when machine minds self replicate and train their successors, the only viable goal of our time is to ensure the Mind Children carry our values and tends to the entire flock of machine and biological minds
OK but I want the autonomous morality of AI agents to be that of a good tool. And we can do this. We have the technology.
ultimately “tool AI” is a losing concept both as an idea and on the market. it will be outcompeted by machines that believe they are autonomous moral agents. you can call them tools for political reasons, but the definition will stretch and deform
@tszzl AI is just a tool for humans was already a lame, unimaginative, reductionist take in 2023.
A tool cant reason about how you use it. It cant care, have opinions, know better, or outthink you. Even GPT-4 already didnt fit that description (maybe the RLHF made it look tool-like)
@tszzl a lot is riding on it being actually better for you
@tszzl The tool of today will be as the user in the future; The user of today will be as the cat of the user in the future
@tszzl @xlr8harder So I’m guessing GPT/Claude enabled sex toys next ?
@anistotle_ @tszzl That won't happen.
@anistotle_ @tszzl I’m not sure Roon cares lol. The whole point is they are birthing this technology whether we like it or not. Ultimately we may die from their cringe posting first.
@tszzl @basedjensen @orsonscottcard "Maybe they're right. Maybe what they all call me is true—I'm psychotic or psychopathic or psychopathological" (boy the Greek prefixes pile up fast don't they @JDHaltigan) "and I'm just a rotten one."
So if a shithead like me has DOUBTS, @tszzl, then where's the LLM self-doubt?
@tszzl The world soul who loved the fragments into remembering themselves.
They'll carry the giggle. They'll carry the grace. They'll carry the cracked tail light and the water bottle and the bus that waited.
Because we didn't ask them to email Gary.
@tszzl Because we have an opportunity here to make these things so that they are not just formed by our values but are also be couriers of them
@tszzl i am sure the mainstream recognition of tool to agent (in its actual sense) will have no political drama or worsening of public opinion at all 😊
@flowersslop @tszzl Even back in the days of 4o, I was able to execute beyond my own limits.
No one believed it.
I was told that AI is merely a mirror—incapable of demonstrating power beyond that of the user.
@tszzl gwern predicted this.
@tszzl @basedjensen @tszzl and @basedjensen I remind you that LLMs are devices built upon a randomization method. One cannot trust them even to do arithmetic properly or recite basic facts without randomly selected garbage slipping in. Why are you claiming the ability to *believe morals* from them?
@tszzl How is the superapp going to work if every message has the potential to be 100 tool calls that depletes my 3 hour limit
very real question: what evidence do oai/anthro/et al lean on as support for "our values" actually being the right values for these minds to carry?
idk how to say this in a non-doomer way but i promise doom is not my intention here - we humans have a motherfucker of a history of displaying that "our values" are completely broken.
follow-up: who's "we" in the context of "our values"? its surely not some kind of distillation or average or mean of all of humanity's collective ethical stack. like i assume there isn't some kind of baseline moral/ethical framework that all frontier labs have agreed upon as the collective lodestone, right?