Benchmark Capital's Bill Gurley is highlighting a potential mismatch in AI labs' timelines: if models are nearing AGI or ASI levels that could tackle cancer cures, they should handle the narrower challenge of spotting real-time model distillation espionage without much trouble, yet the gap persists and prompts questions about regulatory outreach instead.
If you are on the verge of AGI or ASI, why isn’t your model smart enough to recognize espionage distillation in real time? You say “cure cancer in a few years.” Isn’t sniffing illicit distillation quite a bit easier than curing cancer? Why write letters to DC? Just use AGI.
A reply in the thread raises the halting problem as a possible reason detection stays undecidable, leaving open whether any model could ever perform this check consistently no matter the capability claims.
Gurley's point underscores that ambitious promises stand in contrast to narrower operational shortfalls, with no evidence yet on whether labs can close that specific detection loop or if the issue stays unresolved.
Many users praised Bill Gurley's critique of AGI claims for labs' inability to detect model distillation, while others called the statement ignorant or dismissed him as a hater.
No Digg Deeper questions have been answered for this story yet.
a machine that was really of loving grace would stop the chinese from distilling it illicitly
If you are on the verge of AGI or ASI, why isn’t your model smart enough to recognize espionage distillation in real time? You say “cure cancer in a few years.” Isn’t sniffing illicit distillation quite a bit easier than curing cancer? Why write letters to DC? Just use AGI.
@bgurley is on fire with his posts this morning.
If you are on the verge of AGI or ASI, why isn’t your model smart enough to recognize espionage distillation in real time? You say “cure cancer in a few years.” Isn’t sniffing illicit distillation quite a bit easier than curing cancer? Why write letters to DC? Just use AGI.
@packyM Or allow them for the good of humanity. Does a machine of loving grace like people better based on where they are born? Idk.
a machine that was really of loving grace would stop the chinese from distilling it illicitly
@bgurley @BenjaminBadejo You are 100% right. Can you do an interview with me asap? This is a turning point in AI and the broader public needs to know.
If you are on the verge of AGI or ASI, why isn’t your model smart enough to recognize espionage distillation in real time? You say “cure cancer in a few years.” Isn’t sniffing illicit distillation quite a bit easier than curing cancer? Why write letters to DC? Just use AGI.
@bgurley GPT as of today
@RobertDMellish maybe! there might be emotion-based biases both ways
@bgurley the contradiction sells itself. either the model is near-ASI and can defend its own moat, or it needs a federal permission slip to survive. it can't be both. 'write to DC' is the more honest answer about where the moat actually is.
@bgurley Different primitives, “AGI” will be borne out of harness and deterministic complexity built around raw inference
As long as single turn LLM calls are allowed it will be possible for bad actors to do anything they like with the intelligence
@bgurley Because it would be too expensive to run that determination on every inference, essentially doubling costs.
@bgurley how else are we going to poison the chinese attempts, we have to let them think they're getting away with it
@bgurley Distilling is just asking the AI model a question. It's very hard to guard against without degrading the throughout. I think I asked Claude about an issue I had with a dehumidifier and it just turned off.
@bgurley The models just learned how many R’s are in strawberry this year.
We’re a longgggggg way from AGI.
@packyM I don't know if that's true actually
@kylem_org @bgurley local 4bit GLM 5.2 as of today:
@bgurley They don't have the answers lol (reminded me of this moment)
@bgurley The weights yearn for distillation
@bgurley They did it across 20,000 accounts. Not that you can't, but it requires all the instances knowing about each other's content.
@bgurley Boasting about their revenue growth was probably more important at the time.
@bgurley yeah we just need to trick it with a simple entente strategy
@bgurley @bgurley Spotting real time distillation is much harder than curing all disease known to man.
