/Tech15h ago

DeepMind Researcher Predicts Networks Of Neural Networks Over Pure Scaling

10141119767.9K

Original post

Fwiw - I understand that this is the concensus view, but I think history will look back with surprise that it didn't bear out in the end.

In the 1960s, an employee at IBM or Bell Labs would have said the same thing about the Mainframe computer... and they were incredible (and many are still in use today).

But it wasn't just "bigger mainframes forever" anymore than the library was just "bigger library of alexandria forever".

I say that as someone who joined DeepMind nearly 10 years ago doing language modeling research. I have had access to large scale and small scale compute during that time.

I personally think there's an enormous amount of low-hanging fruit which doesn't require.

The future is networks of neural networks: - better routers - better benchmarks - better access to non-public / niche information - better pricing mechanisms - better source attribution - better unlearning - ...

There's so much great research to be done. And much of it remains low-hanging because there are some subtle reasons why highly resourced orgs don't tend to pursue them.

Aidan Clark@_aidan_clark_

If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup.

The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.

9:10 PM · Jun 9, 2026 · 67.9K Views

/Tech15h ago

DeepMind Researcher Predicts Networks Of Neural Networks Over Pure Scaling

10141119767.9K

#366

Original post

⿻ Andrew Trask@iamtrask

Fwiw - I understand that this is the concensus view, but I think history will look back with surprise that it didn't bear out in the end.

In the 1960s, an employee at IBM or Bell Labs would have said the same thing about the Mainframe computer... and they were incredible (and many are still in use today).

But it wasn't just "bigger mainframes forever" anymore than the library was just "bigger library of alexandria forever".

I say that as someone who joined DeepMind nearly 10 years ago doing language modeling research. I have had access to large scale and small scale compute during that time.

I personally think there's an enormous amount of low-hanging fruit which doesn't require.

There's so much great research to be done. And much of it remains low-hanging because there are some subtle reasons why highly resourced orgs don't tend to pursue them.

Aidan Clark@_aidan_clark_

If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup.

The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.

9:10 PM · Jun 9, 2026 · 67.9K Views

Sentiment

Positive users are optimistic that networks of specialized neural models will deliver higher accuracy and easier safety, while negative users dismiss pretrain labs as outdated and demand personal AGI on cellphones.

Pos

33.3%

Neg

66.7%

6 comments with sentiment.

Cluster Engagement

Posts from X

Most Activity

VIEWS1.1K

⿻ Andrew Trask@iamtrask

If @peterthiel asked me "what does everyone believe that I know to be wrong"... this would be my answer.

(And Aidan is very smart and he's describing a view for which there is plenty of evidence, and near universal agreement. I just disagree.).

1d1.1K7

BOOKMARKS9LIKES8

⿻ Andrew Trask@iamtrask

If none of this makes sense and you just need to read "a completely different way to think about AI progress" and a bunch of launch points for research in AI, a few links: - https://attribution-based-control.ai/ - https://github.com/iamtrask/abcGPT - https://openmined.org/blog/what-is-broad-listening/ - - https://openmined.org/blog/secure-enclaves-for-ai-evaluation/

1d1.1K89

RETWEETS10

⿻ Andrew Trask@iamtrask

Fwiw - I understand that this is the concensus view, but I think history will look back with surprise that it didn't bear out in the end.

In the 1960s, an employee at IBM or Bell Labs would have said the same thing about the Mainframe computer... and they were incredible (and many are still in use today).

But it wasn't just "bigger mainframes forever" anymore than the library was just "bigger library of alexandria forever".

I say that as someone who joined DeepMind nearly 10 years ago doing language modeling research. I have had access to large scale and small scale compute during that time.

I personally think there's an enormous amount of low-hanging fruit which doesn't require.

There's so much great research to be done. And much of it remains low-hanging because there are some subtle reasons why highly resourced orgs don't tend to pursue them.

Aidan Clark@_aidan_clark_

If you want to work on pretraining-for-AGI, join OpenAI, Google, Meta or the Anthropic/XAI/Cursor supergroup.

The bitter truth of the widening compute gap is that all the problems which are actually on the critical path to AGI now demand that level of compute.

1d67.9K14197

REPLIES1

broadfield-dev@broadfield_dev

@iamtrask wouldn't. Those companies are at least 12 months behind SOTA. Code is finite, predictable, and comes with error messages. They will never build anything but great copy/paste databases.

They are just big.

1d124

⿻ Andrew Trask@iamtrask

@peterthiel To be more specific. A global network of highly interconnected, neural-network routed, small, specialized models will ultimately deliver: - higher accuracy - faster speed - lower cost

than large, monolithic systems.

1d66162

⿻ Andrew Trask@iamtrask

One final thing - I think the biggest barrier to breakthrough research is allowing yourself to subscribe to industry groupthink (or to the polar opposite of that groupthink).

Go in a 3rd direction. Follow the scaling laws. Look for bridges across fields (especially deep learning, cryptography, and distributed systems).

It's never been a better time to do research.

1d86081

⿻ Andrew Trask@iamtrask

@peterthiel They'll also be easier to make safe in many ways, but now I'm going on a tangent. This is enough for now.

1d3326

Joel Kreager@JoelKreager

@iamtrask Can AI really break reality any better than QAnon did? We can't agree on the most basic facts already.

15h481

dragonAI@AIMLforEdu

@iamtrask What does it mean by networks of neural networks? 🤔

19h391

tsunami_crypto@ls_brd

@iamtrask so he thinks the pretrain-on-mainframes crowd is just slow to see whats obvious

name one frontier lab where the bitter truth is not hiring

1d97

Fergus Meiklejohn@airuyi

@iamtrask I'm doing work on my macbook pro, I'm on CPU now. But I do use the models to write code and if model developers nerf them that will make work more difficult I guess. The big issue as ever is money. There are massive IPOs coming and owners want lots of money.

21h92

broadfield-dev@broadfield_dev

@iamtrask Personal AGI on your cellphone or gtfo.

1d7