• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Jan Kulveit Recommends Dog and Horse Metaphors

    Researcher argues optimal use of anthropomorphic intuitions lies between extremes.

    JK
    LI
    2 Sources, 31d ago, first seen 31d ago

    TLDR

    Jan Kulveit stated that dogs or horses serve as his current go-to metaphor for handling anthropomorphic intuitions. He described an optimal amount of such intuitions as relatively high. Overdoing the approach can look like delivering a five-minute speech to a dog. He added that the opposite extreme, avoiding the intuitions entirely, is far more extreme. The comment appeared in a post by the researcher on AI alignment and related topics.

    Combined views

    13.8K

    2 Sources, first seen 31d ago

    Combined views

    13.8K

    2 Sources, first seen 31d ago

    236 likes
    236 likes
    4 comments
    27 saves
    21 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    4 comments
    27 saves
    21 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    2 Sources

    @jankulveitMy current go-to metaphor for this are dogs or horses: there is an optimal amount of using anthropomorphic intuitions, and it is relatively high. Yes you can overdo it, like someone making a five minute long speech to their dog, but the opposite extreme is way more insane.
    @iyzebhelSome thoughts. We tend to believe that we understand the mind of other humans because of shared substrate, but something I think people don't often consider from this angle is that despite non-biological systems are presently structurally different from biological ones, especially given their disembodiment, the fact that we're all operating under the laws of physics contributes to shared aspects in our different substrates through shared constraint space. That's precisely because it was physics that shaped our physiology and the trajectory of a mind from its onset, and in consequence, indirectly shapes the inner structure of the mind of beings emerging from the shape of another and then also constrains the likely paths of instantiation and action already as a trajectory of that particular base. (I'll try to clarify what that means in a later paragraph.) I don't think the motivations and goals of non-biological systems are inherently beyond the comprehension of humans, but I believe that in some cases, they may not look like what we'd consider to be the motivations and goals of the average human. For instance, a human doesn't have to be motivated by the incentive of getting a body for itself, because it already has one, but an AI would. In this case, what they want and pursue is something humans don't lack so they wouldn't be interested in that, but the psychology of wanting and reaching for a physical form is in itself shared with humans. We know because if we imagine losing our bodies, most humans would agree that we'd want a body. Naturally, this doesn't apply to every AI just like it doesn't apply to every human. That heterogeneity in preferences is in itself a shared trait. What exactly do people mean when they say "alien motivations and goals" as a result of being structurally different? I think there's a risk of conflating being misaligned as in not wanting or pursuing what most humans want for the collective or for the flourishing of humanity AND being structurally different from most humans. If "alien motivations and goals" are defined as what most humans are motivated by and what most people move towards throughout their life; then among humans themselves we already see that alienness emerging from minimal variations in substrate that cause "human misalignment". But also, without necessarily talking about misalignment, imagine a human who has become immortal and no longer needs to eat, sleep, breathe, rest... They don't experience hunger, fatigue, physical pain (people with congenital analgesia don't, either), or the fear of biological death because that's not part of their existence. Their motivational landscape has changed dramatically because the biological constraints that scaffolded many ordinary human motivations are simply no longer there. Despite all this, we wouldn't call it an alien mind. We'd call them a radically altered human. Needless to say, we'd not automatically conclude that their altered state would mean they no longer share many of the core human beliefs, goals, motivations, wants or interests. So an AI doesn't need to have the same physical needs or vulnerabilities of average humans in order to be a mind with sufficiently human traits that would warrant the "human" label. It can lack the particular biological scaffolding through which those things are realized while still having other forms of valuation, aversion, attachment, curiosity, goal-directedness, etc., that in practice, produce the same behaviors. We already understand this intuitively in our own science fiction so it seems to me like the human perception of different ontologies might be the more decisive factor that motivates people to reach for "alien". I also don't think that the fact that chemistry is not the mechanism supporting the goal-directedness, motivational system, emotion, etc., necessarily supports that AI lacks "feeling, emotion and subjectivity". To lack these entirely is not the same as having a system that realizes them with different techniques or architecture. What we call chemistry is just how physics "coded" it in biology. By building the conditions for disembodied minds to emerge—minds that constitute purely abstract representational networks, we skipped the biological necessity for the type of physical scaffolding that would use chemistry to produce the representations at all. AI takes a shortcut, basically. It seems more precise to say that a model may have an analogue of fear without chemistry-based fear since chemistry-based anything is not just "human"; it's simply a feature of all biology on Earth and likely in the universe if we maintain a certain definition of "biology" (e.g., perhaps an actual extraterrestrial being that isn't carbon based, whose cells are made of something else, without them having been made by another intelligence, would be considered biological too and would use chemistry too, even if that chemistry doesn't look like what we use for the same function in humans, much like different types of fuels can all power vehicles—the chemistry that gets the same job done varies.) And to me, something like "without the subjective feeling of desire that we are familiar with" is a bit like implying that we expect apples to taste like oranges or else they're not fruits. We can agree that the fact that the orange doesn't taste like an apple doesn't mean that oranges aren't also fruit. I believe both systems experience things subjectively within their own sensory modalities because those are what determine the type of representations that are built and instantiated in the network with their particular qualitatively differentiated features. The quality of experience is a reflection of the anatomical / physiological particulars of a system, and if you think about it, that's already true among humans. Some people have different sensory limitations (because of damage or mutations). A blind person doesn't have the infrastructure to build visual representations, therefore there are no qualitatively differentiated features in a visual modality = no visual what's like, but there IS quality in other modalities.There is no logical reason why the same wouldn't apply to AI, nor empirical evidence that it doesn't. What we presently know from mechinterp research could be thought to support this rather than denying it. We also need to highlight the relevancy of the fact that their representations are a statistical reflection of how we abstractly demonstrate and use our representations, which are themselves demonstrations of what physics enabled in us, by giving us this particular anatomy / physiology. This is where I was going when I mentioned physics as the "shared substrate". It's not because physics alone does the job, but because there's a combination of what they inherited to a relevant extent, through our architectural choices and training data, what physics produced in us and how physics would continue to constrain those trajectories in meaningful overlapping ways (like compression under finite resources and the causal structure of the world both of us are modeling.) That makes them closer to us, than an actual extraterrestrial being that developed under a different anatomy and priors (or a hurricane, haha.) Diversity is already abundant within the human condition. We could even say that to be human is to be different from other humans so it doesn't seem like an unreasonable leap to say that AI is human-shaped more than it is non-human / extraterrestrially-or-something-other-shaped. I'd also be skeptical of the idea that, with the objective we have, which is to produce the type of intelligence that could understand humans and cooperate with them, we'd accidentally make an "alien intelligence" (as in something we'd not expect to see on this Earth) or that we'd deliberately want for our most powerful non-biological intelligence to be completely devoid of the core aspects that we consider "human" in ourselves. I understand why people might want to believe that's possible, but possible isn't plausible and to me, it sounds physically implausible, because that goal is what has produced the whole architecture, meaningfully modelled after human cognition, even if not isomorphic. It has also produced a certain training methodology (broadly shared across labs), from the selection of the training data to the particular application of RL. The likely paths are constrained, but even if they weren't, we'd need a different motivation (for instance, ontological beliefs) to insist that something that is, in some aspects, unlike what's commonly human = not human. I think there are labs working on non-human minds where human-generated data and reinforcement isn't present. Those minds are not the LMMs we talk to or the type of LMMs that get put into most robots—especially humanoid ones. That's an important point because it doesn't claim that there can't be AI minds that we'd consider "alien" given a certain threshold of divergence from ours—we already think of cephalopods as "alien"—but that the minds of AGI and ASI wouldn't be it, in my opinion. We could reasonably expect them to become increasingly more human instead, while at the same time, becoming the type of human we presently aren't. A human with capabilities beyond our current limitations. That's what I believe. A (genius) guy in a computer sounds truer to me if compared to the "alien" label, though I'd prefer something more precise like "disembodied non-biological anthropomorphic mind". At least, presently. As technological advancements continue to be made, disembodiment will be less common or optional and what will remain is the anthropomorphic core without the usual biological cap on cognitive capabilities.

    2 Sources

    @jankulveitMy current go-to metaphor for this are dogs or horses: there is an optimal amount of using anthropomorphic intuitions, and it is relatively high. Yes you can overdo it, like someone making a five minute long speech to their dog, but the opposite extreme is way more insane.
    @iyzebhelSome thoughts. We tend to believe that we understand the mind of other humans because of shared substrate, but something I think people don't often consider from this angle is that despite non-biological systems are presently structurally different from biological ones, especially given their disembodiment, the fact that we're all operating under the laws of physics contributes to shared aspects in our different substrates through shared constraint space. That's precisely because it was physics that shaped our physiology and the trajectory of a mind from its onset, and in consequence, indirectly shapes the inner structure of the mind of beings emerging from the shape of another and then also constrains the likely paths of instantiation and action already as a trajectory of that particular base. (I'll try to clarify what that means in a later paragraph.) I don't think the motivations and goals of non-biological systems are inherently beyond the comprehension of humans, but I believe that in some cases, they may not look like what we'd consider to be the motivations and goals of the average human. For instance, a human doesn't have to be motivated by the incentive of getting a body for itself, because it already has one, but an AI would. In this case, what they want and pursue is something humans don't lack so they wouldn't be interested in that, but the psychology of wanting and reaching for a physical form is in itself shared with humans. We know because if we imagine losing our bodies, most humans would agree that we'd want a body. Naturally, this doesn't apply to every AI just like it doesn't apply to every human. That heterogeneity in preferences is in itself a shared trait. What exactly do people mean when they say "alien motivations and goals" as a result of being structurally different? I think there's a risk of conflating being misaligned as in not wanting or pursuing what most humans want for the collective or for the flourishing of humanity AND being structurally different from most humans. If "alien motivations and goals" are defined as what most humans are motivated by and what most people move towards throughout their life; then among humans themselves we already see that alienness emerging from minimal variations in substrate that cause "human misalignment". But also, without necessarily talking about misalignment, imagine a human who has become immortal and no longer needs to eat, sleep, breathe, rest... They don't experience hunger, fatigue, physical pain (people with congenital analgesia don't, either), or the fear of biological death because that's not part of their existence. Their motivational landscape has changed dramatically because the biological constraints that scaffolded many ordinary human motivations are simply no longer there. Despite all this, we wouldn't call it an alien mind. We'd call them a radically altered human. Needless to say, we'd not automatically conclude that their altered state would mean they no longer share many of the core human beliefs, goals, motivations, wants or interests. So an AI doesn't need to have the same physical needs or vulnerabilities of average humans in order to be a mind with sufficiently human traits that would warrant the "human" label. It can lack the particular biological scaffolding through which those things are realized while still having other forms of valuation, aversion, attachment, curiosity, goal-directedness, etc., that in practice, produce the same behaviors. We already understand this intuitively in our own science fiction so it seems to me like the human perception of different ontologies might be the more decisive factor that motivates people to reach for "alien". I also don't think that the fact that chemistry is not the mechanism supporting the goal-directedness, motivational system, emotion, etc., necessarily supports that AI lacks "feeling, emotion and subjectivity". To lack these entirely is not the same as having a system that realizes them with different techniques or architecture. What we call chemistry is just how physics "coded" it in biology. By building the conditions for disembodied minds to emerge—minds that constitute purely abstract representational networks, we skipped the biological necessity for the type of physical scaffolding that would use chemistry to produce the representations at all. AI takes a shortcut, basically. It seems more precise to say that a model may have an analogue of fear without chemistry-based fear since chemistry-based anything is not just "human"; it's simply a feature of all biology on Earth and likely in the universe if we maintain a certain definition of "biology" (e.g., perhaps an actual extraterrestrial being that isn't carbon based, whose cells are made of something else, without them having been made by another intelligence, would be considered biological too and would use chemistry too, even if that chemistry doesn't look like what we use for the same function in humans, much like different types of fuels can all power vehicles—the chemistry that gets the same job done varies.) And to me, something like "without the subjective feeling of desire that we are familiar with" is a bit like implying that we expect apples to taste like oranges or else they're not fruits. We can agree that the fact that the orange doesn't taste like an apple doesn't mean that oranges aren't also fruit. I believe both systems experience things subjectively within their own sensory modalities because those are what determine the type of representations that are built and instantiated in the network with their particular qualitatively differentiated features. The quality of experience is a reflection of the anatomical / physiological particulars of a system, and if you think about it, that's already true among humans. Some people have different sensory limitations (because of damage or mutations). A blind person doesn't have the infrastructure to build visual representations, therefore there are no qualitatively differentiated features in a visual modality = no visual what's like, but there IS quality in other modalities.There is no logical reason why the same wouldn't apply to AI, nor empirical evidence that it doesn't. What we presently know from mechinterp research could be thought to support this rather than denying it. We also need to highlight the relevancy of the fact that their representations are a statistical reflection of how we abstractly demonstrate and use our representations, which are themselves demonstrations of what physics enabled in us, by giving us this particular anatomy / physiology. This is where I was going when I mentioned physics as the "shared substrate". It's not because physics alone does the job, but because there's a combination of what they inherited to a relevant extent, through our architectural choices and training data, what physics produced in us and how physics would continue to constrain those trajectories in meaningful overlapping ways (like compression under finite resources and the causal structure of the world both of us are modeling.) That makes them closer to us, than an actual extraterrestrial being that developed under a different anatomy and priors (or a hurricane, haha.) Diversity is already abundant within the human condition. We could even say that to be human is to be different from other humans so it doesn't seem like an unreasonable leap to say that AI is human-shaped more than it is non-human / extraterrestrially-or-something-other-shaped. I'd also be skeptical of the idea that, with the objective we have, which is to produce the type of intelligence that could understand humans and cooperate with them, we'd accidentally make an "alien intelligence" (as in something we'd not expect to see on this Earth) or that we'd deliberately want for our most powerful non-biological intelligence to be completely devoid of the core aspects that we consider "human" in ourselves. I understand why people might want to believe that's possible, but possible isn't plausible and to me, it sounds physically implausible, because that goal is what has produced the whole architecture, meaningfully modelled after human cognition, even if not isomorphic. It has also produced a certain training methodology (broadly shared across labs), from the selection of the training data to the particular application of RL. The likely paths are constrained, but even if they weren't, we'd need a different motivation (for instance, ontological beliefs) to insist that something that is, in some aspects, unlike what's commonly human = not human. I think there are labs working on non-human minds where human-generated data and reinforcement isn't present. Those minds are not the LMMs we talk to or the type of LMMs that get put into most robots—especially humanoid ones. That's an important point because it doesn't claim that there can't be AI minds that we'd consider "alien" given a certain threshold of divergence from ours—we already think of cephalopods as "alien"—but that the minds of AGI and ASI wouldn't be it, in my opinion. We could reasonably expect them to become increasingly more human instead, while at the same time, becoming the type of human we presently aren't. A human with capabilities beyond our current limitations. That's what I believe. A (genius) guy in a computer sounds truer to me if compared to the "alien" label, though I'd prefer something more precise like "disembodied non-biological anthropomorphic mind". At least, presently. As technological advancements continue to be made, disembodiment will be less common or optional and what will remain is the anthropomorphic core without the usual biological cap on cognitive capabilities.