Existing memory techniques remain largely unused in popular LLMs.
i post about this a lot
seems big AI labs are hyperfixating on reasoning when they should focus on *memory* instead
normal people won't use models that can think for hours to solve hard math problems
people want models that learn over time, remember details, adapt and interact like a person would
Users are impressed by the depth of work on memory-native LLMs and continual learning approaches, and thank researchers advancing these methods for AI.
No Digg Deeper questions have been answered for this story yet.
Great team building some cool stuff!
The single biggest impediment in AI isn't absolute intelligence or coding ability. It's deeply understanding large repositories of knowledge that every person and every company has.
Our repositories will explode in size, with AI agents writing much of them.
This is a two-way street: we need to understand what our AI generates, and AI needs to better understand us. A breakthrough in memory is needed – gradients need to be taken, and our team is the right one to take the big swing.
The single biggest impediment in AI isn't absolute intelligence or coding ability. It's deeply understanding large repositories of knowledge that every person and every company has.
Our repositories will explode in size, with AI agents writing much of them.
This is a two-way street: we need to understand what our AI generates, and AI needs to better understand us. A breakthrough in memory is needed – gradients need to be taken, and our team is the right one to take the big swing.
http://x.com/i/article/2069463677733142528
Dan saw all the models doing this, and he took that personally
@dan_biderman is an amazing researcher and leader. I'm amazingly proud to call him my former intern.
He has assembled a world-class team, and I truly believe they have the ability and resources to transform the way we build relationships with language models.
also in general i've been really impressed with the amount of DEPTH there is to building models in industry, and working on larger projects over longer timespans
honestly the first model we trained was not very smart. a few clever ideas and so many little fixes/improvements later, the model is surprisingly intelligent, and i talk to it all the time
it feels good
@dan_biderman is an amazing researcher and leader. I'm amazingly proud to call him my former intern.
He has assembled a world-class team, and I truly believe they have the ability and resources to transform the way we build relationships with language models.
The single biggest impediment in AI isn't absolute intelligence or coding ability. It's deeply understanding large repositories of knowledge that every person and every company has.
Our repositories will explode in size, with AI agents writing much of them.
This is a two-way street: we need to understand what our AI generates, and AI needs to better understand us. A breakthrough in memory is needed – gradients need to be taken, and our team is the right one to take the big swing.
congrats on launch! 🌱🌻
excited to see all the fun takes on memory / continual learning
you can rag, you can compact, you can continuously rl / sft on updated data (kinda like the days of “old”)
you can collect more reasoning traces, and create new envs
but friends here are exploring new ideas w new memory components + new ways to train them
also could not have had better day-1 labmates @EyubogluSabri @MayeeChen @NeelGuha @dan_biderman (last one maybe day-1460 but who’s counting)
http://x.com/i/article/2069463677733142528
@mzhangio Thank you Michael!
@code_star wait this actually looks clean
