Gael Varoquaux warns that abandoning reductionism risks anecdotal science
A fixation on system reductions (e.g., analyzing implicit biases of SGD in a two layer linear network) has been a tremendous setback in understanding DL, because they are not analogous with the systems we want to understand. Few realize we can analyze the models we actually use.
@andrewgwils Reductionism and analyzing complex objects has always been a chicken and egg problem: as long as we haven't found the right abstractions for reduction, we're failing to get generalization. But science without reduction has the danger of being a collection of anecdotes
A fixation on system reductions (e.g., analyzing implicit biases of SGD in a two layer linear network) has been a tremendous setback in understanding DL, because they are not analogous with the systems we want to understand. Few realize we can analyze the models we actually use.
Gael Varoquaux warns that abandoning reductionism risks anecdotal science
A fixation on system reductions (e.g., analyzing implicit biases of SGD in a two layer linear network) has been a tremendous setback in understanding DL, because they are not analogous with the systems we want to understand. Few realize we can analyze the models we actually use.