Kalomaze argues highly compressed models lack deep learning impact.
@jsuarez @stalkermustang i.e. "AlphaGo wouldn't have been very impressive at ~50k params"
@jsuarez @stalkermustang if you can in principle write GOFAI style symbolic solvers for most of the tasks being trained for 'superhuman' nano resnets, then its like. not really exploring the "deep" or "neural" parts, is all
No Digg Deeper questions have been answered for this story yet.
AlphaGo would have been much more impressive at 50k params. It would only have been less impressive if they got it that small by introducing a bunch of domain specific hacks (fwiw I don't think you can do Go wellin 50k). We are not introducing hacks to get down to 50k. It's just that we can run as many samples on 1 node with our setup as OAI/DM were running on 100 back in ~2020
@jsuarez @stalkermustang i.e. "AlphaGo wouldn't have been very impressive at ~50k params"