DeepSeek's Deli Chen releases an LLM continual learning survey paper generated autonomously by the DeliAutoResearch system
It maps out self-improvement methods like STaR and o1.
It's actually pretty cool that Deli is moving the Overton window all by himself. This is not "AI slop", the quality is well above a naive LLM generation or even some "deep research" report that often gets passed for a legit human paper. We should openly share AI surveys.
Deli Chen@victor207755822The Second Paper from DeliAutoResearch SKILL is here! 🎉✨ https://victorchen96.github.io/continual_learning_survey.pdf This time the focus is on continual learning and self-iteration, which I believe is an absolutely essential step on the road to AGI. 🧠💡 Future AI systems definitely won't be frozen sets of parameters — they'll be systems that can self-iterate and keep evolving. 🌱🔄 Context management and documented memory can help a model sustain attention, but the attention window will eventually get filled up. When that happens, you need to parameterize knowledge and experience to reduce the cognitive load. 📚🧸 Also, the (simulated) peer review score hit an 8 this round — a huge leap from the 6 last time! 🚀🐣 In this paper, the model tried calling advanced language models to design and run experiments all on its own for the very first time — a capability it didn't have before. 🤖🌸 Btw, the second image includes a production data comparison between Paper 2 and Paper 1. In a nutshell, as the skill itself iterated, the number of interaction rounds dropped dramatically, while the total tokens consumed went up significantly — which is actually a really good sign! It shows SKILL is shifting toward greater autonomy. 📊🐱💕 Really looking forward to the day soon when DeliAutoResearch SKILL can produce truly master-level academic writing. 🎓🍀🌟
it is ironic that the paper about continuous learning is obviously biased towards bits preceding the agent's knowledge cutoff. Virtually nothing from 2025-2026 made the cut.
Teortaxes▶️ (DeepSeek 推特🐋铁粉 2023 – ∞)@teortaxesTexIt's actually pretty cool that Deli is moving the Overton window all by himself. This is not "AI slop", the quality is well above a naive LLM generation or even some "deep research" report that often gets passed for a legit human paper. We should openly share AI surveys.
Since arXiv doesn't allow AI to be listed as authors, I had to move DeepSeek-V4-Pro and GPT-Image-2 from the author line to the footnotes — even though they literally did 99% of the work on this paper. 🤖🐣✨📝
Deli Chen@victor207755822The Second Paper from DeliAutoResearch SKILL is here! 🎉✨ https://victorchen96.github.io/continual_learning_survey.pdf This time the focus is on continual learning and self-iteration, which I believe is an absolutely essential step on the road to AGI. 🧠💡 Future AI systems definitely won't be frozen sets of parameters — they'll be systems that can self-iterate and keep evolving. 🌱🔄 Context management and documented memory can help a model sustain attention, but the attention window will eventually get filled up. When that happens, you need to parameterize knowledge and experience to reduce the cognitive load. 📚🧸 Also, the (simulated) peer review score hit an 8 this round — a huge leap from the 6 last time! 🚀🐣 In this paper, the model tried calling advanced language models to design and run experiments all on its own for the very first time — a capability it didn't have before. 🤖🌸 Btw, the second image includes a production data comparison between Paper 2 and Paper 1. In a nutshell, as the skill itself iterated, the number of interaction rounds dropped dramatically, while the total tokens consumed went up significantly — which is actually a really good sign! It shows SKILL is shifting toward greater autonomy. 📊🐱💕 Really looking forward to the day soon when DeliAutoResearch SKILL can produce truly master-level academic writing. 🎓🍀🌟