Reactions from ranked influencers
4 posts
Coming up with a complicated new neural network architecture and measuring its improvement not on wallclock, but on overall trainer steps against an unturned baseline to get a 3% gain which could have been done by deleting one bad piece of data mixed into the training corpus
the things that will probably improve your policy more than any cool new science or technical innovation ever could: - cleaning the data yourself - using more parameters - improving sensor placement (in robotics)
Coming up with a complicated new neural network architecture and measuring its improvement not on wallclock, but on overall trainer steps against an unturned baseline to get a 3% gain which could have been done by deleting one bad piece of data mixed into the training corpus
there is no Glory in cleaning data (look ma! i removed artifacts from broken machine translated text! that's cute, sweetie, but Anyone Can Do That) instead, all the Glory goes to the Breathrough-Chasers of architecture and algorithmic novelty for Future Lightcone Flagplant Purposes (fame! power! money! adoration from the parasocial masses!)
Coming up with a complicated new neural network architecture and measuring its improvement not on wallclock, but on overall trainer steps against an unturned baseline to get a 3% gain which could have been done by deleting one bad piece of data mixed into the training corpus
@suchenzang someone doing data work is the biggest green flag