A 2022 arXiv paper titled 'General Intelligence Requires Rethinking Exploration' calls for shifting from learning from data to learning what data to learn from and new methods beyond reinforcement learning

Nice of @jennyzhangzt to share this paper, which I selfishly think was ahead of its time. The context was that I was leaving Meta to do another startup, and thought I would not be writing papers for years. Of course, @MinqiJiang had all the good ideas + did most of the writing 😅

time to rethink exploration

https://arxiv.org/abs/2211.07819

@willccbb I think as long as the model can't learn the concept of time, there will be no real exploration ( which BTW is why agent frontend sw thrives now, it's basically a conduit for LLM to borrow clock from the CPU). And since time points to entropy increase, all ce loss min won't work

was reading a paper last night that felt very timely and refreshing, like the sort of thing that was bound to anchor the next wave of innovation in self-teaching

then i realized it was from 2022

@willccbb it's time to explore

@willccbb Cathedrals everywhere

@willccbb nice find. Adding Whiteboard summary (ChatGPT image gen)

@willccbb How is this different from Unsipervised Environment design? In RL there are already lots of papers and i assume some folks already managed to apply ideas from UED. Seems like this paper is just the same findings albeit from other perspective

@egrefen @jennyzhangzt Thanks for the kind words (and for originally instigating this paper)!

Talking through and rethinking these ideas together over the ~6 months spent writing this was the most transformative part of my PhD, when I figured out to a great extent what I believed as a researcher.

many such cases

@willccbb Ref?

@egrefen @MinqiJiang definitely ahead of its time

@willccbb wandering vs exploring

@willccbb mmmm

@willccbb AI years are so cursed that 2022 already feels like ancient literature

@MinqiJiang @willccbb And rethink it.

@brandonsdinunno

@willccbb Was all llm fever after that

@willccbb Ah the open endedness bait.

@maxsbob21 @willccbb @MinqiJiang (the first author) is a pioneer of UED (was also the first author on PLR, for example)