Reaction
Principled credit assignment as a possible breakthrough in recursive AI self-improvement
A user argues ultra-large, asynchronous training needs something stronger than “backprop on stilts” to assign credit.
TLDR
A user predicts that solving principled credit assignment for ultra-large, asynchronous training will be a breakthrough in recursive AI self-improvement. In a reply to their own post, they add that training 100T MoEs with “dumb routers” is possible, but would feel wasteful to them.
Combined views
740
2 Sources, first seen ago