I feel like the obsession with continual learning / sample efficiency leads the field in the wrong direction. It's the bad career strategy of focusing on addressing your weaknesses instead of maximizing your strengths.

Yes, there is an existence proof in the human brain, but it doesn't by any means guarantee that that'll be the most interesting AI. It may require $100T of R&D on chips and AI methods to get that unlock.

On the other side of things, it's obvious that the coming models are extremely transformative and built on technologies that we already have. There's great reason to focus on just maximizing this. In reality, this is what the frontier labs are doing. They're going as fast as possible down the current development tree. This is good for progress and mixed for safety/geopolitics.

Things like "automate white color work" and "replace the AI researcher job" are the guesses of labs because it's super hard to imagine futures for what these dramatic technologies will be. Don't take the labs too seriously about this being the exact goal. The exact goal is to push the frontier and monetize later.

Solving continual learning, sample efficiency, etc would be great, but its trying to predict when a scientific breakthrough will come instead of trying to grapple with how the 100% sure thing coming technological revolution will change our lives.

This isn't to say the Dwarkesh post is bad, it addresses some reasonable critiques, but it is the least bitter lesson pilled thing to be obsessed with human intelligence and how that can inform AI.

We are in the AGI era of research. This is about embracing the unknown, scaling resources, and seeing what is enabled by making a series of magical tweaks to complex recipes that build frontier models. Lean into the alchemy.

(it should be pretty clear that I personally, investing in open research agree we need fundamental science -- just not agreeing that this is what the "cutting edge of the frontier" is governed by)

Here is a thought: LLMs are super sample efficient at learning things in context. It's just that ICL is limited as of now. What we need are mechanisms to somehow absorb the information stored "in-context" into the weights. This is in a sense what fast weights do in GDNs to an extent.

Then the question becomes how do we train LLMs to learn to efficiently pack information into fast weights( and eventually slow weights ) over very long horizons. This is a problem that can be solved if trained on the right reward functions and tasks imo. Maybe just a matter of time.

In that case, we could still think of pre training as rolling up a base arch, similar to how evolution produced the human, brain with the difference that our pre training and post training still has not produced an arch that can do long range in-contex -> fast weights -> slow weights effectively.

Wrote about bit about it here: https://www.aravindjayendran.com/writing/few-shot-learners-cant-remember