/AI4h ago

Researcher Uses Road Trip Metaphor to Clarify Rising AGI Definition Debates

335042.8K
#349Geoffrey Irving
Samuel Hammond 🦉
Original post
Geoffrey Irving@geoffreyirving#349inAI

Here is a metaphor for AGI definitions.

Imagine you’re on a long drive from Los Angeles to the Bay Area (for me: undergrad to grad school). From far away, this is unambiguous: the Bay Area is very small relative to the Los Angeles/Bay Area distance. People can and do dispute what exactly “The Bay Area” is (there are many definitions), but no one in LA would say, “I have no idea what direction you are going”.

But now you approach the actual Bay Area. It’s a vague place! The definitional ambiguity starts to ramp up. If you’re in Los Gatos and you say “I’m driving to the Bay Area”, people will have questions.

If we track the conversation as we drive from LA on, the definitional ambiguity and disagreement will ramp up over time. A skeptic that “the Bay Area” is a coherent idea might look at the ramp and think “aha I was right, people are starting to realize that the concept was incoherent all along”. And indeed, the people with questions are right to ask them, the relative distances have changed, “But where in the Bay Area?” matters more.

But the definitional ambiguity is because we’re getting close! Something is about to happen!

10:17 AM · Jun 6, 2026 · 2.3K Views
Sentiment
Sentiment building, check back later.
Cluster Engagement
Posts from X
Most Activity
Most Activity
VIEWS799BOOKMARKS1LIKES4
Samuel Hammond 🦉@hamandcheese

Vague predicates are pervasive in language. Wittgenstein dealt with them in terms of "family resemblances." Little did he know he was grokking the idea of interpolation within a latent space. We are not at AGI yet, but are close enough to be debating when a grain becomes a heap.

Geoffrey Irving@geoffreyirving

Here is a metaphor for AGI definitions.

Imagine you’re on a long drive from Los Angeles to the Bay Area (for me: undergrad to grad school). From far away, this is unambiguous: the Bay Area is very small relative to the Los Angeles/Bay Area distance. People can and do dispute what exactly “The Bay Area” is (there are many definitions), but no one in LA would say, “I have no idea what direction you are going”.

But now you approach the actual Bay Area. It’s a vague place! The definitional ambiguity starts to ramp up. If you’re in Los Gatos and you say “I’m driving to the Bay Area”, people will have questions.

If we track the conversation as we drive from LA on, the definitional ambiguity and disagreement will ramp up over time. A skeptic that “the Bay Area” is a coherent idea might look at the ramp and think “aha I was right, people are starting to realize that the concept was incoherent all along”. And indeed, the people with questions are right to ask them, the relative distances have changed, “But where in the Bay Area?” matters more.

But the definitional ambiguity is because we’re getting close! Something is about to happen!

1hViews 799Likes 4Bookmarks 1
REPLIES1
Leon Lang@Lang__Leon

@geoffreyirving https://open.substack.com/pub/helentoner/p/the-term-agi-is-almost-useless-at

1hViews 16Likes 1
Kiril Bangachev@BangachevKiril

@geoffreyirving That’s why we decrease learning rate in first order methods! Early steps are much more certain

1hViews 28Likes 1
Leon Lang@Lang__Leon

@geoffreyirving Helen Toner’s version of this:

1hViews 27Likes 1
Geoffrey Irving@geoffreyirving

@BangachevKiril Alas, we are doing the opposite of that.

1hViews 16Likes 1
Geoffrey Irving@geoffreyirving

FWIW I disagree with that picture, and some of the key points made literally in the article, though I expect Helen would quickly agree with the corrections.

1. I do think all sensible definitions of AGI point into the future, so the picture is implying some misleading temporal sequencing. 2. AGI is still very useful as a term! It is not useful because it is crisp and specific, or risk-relevant: specific risks almost all occur prior to AGI or after AGI (superintelligence). 3. The usefulness is that *generality is part of the correct mental model of why AI has been going so fast*. If back in 2022 you decided to strike AGI from your mental vocabulary, you would probably (not necessarily, but probably) have mispredicted the rate of progress.

1hViews 7Likes 1