AI researchers revive visualizations from 2009 LeCun paper
Roland Memisevic posted a 16-by-16 grid of filters from the 2009 paper Learning Invariant Features through Topographic Filter Maps by Koray Kavukcuoglu, Marc'Aurelio Ranzato, Rob Fergus, and Yann LeCun. The images display grids of oriented stripes, curved waves, rings, and edges that illustrate continuous topographic organization of learned features. Sander Dieleman and Luca Ambrogioni identified the source paper and noted its prior influence on neuroscience research.
#4Yann LeCun@YLECUN@pfau @sedielem @koraykv @MarcRanzato @rob_fergus We never heard from them again.
David Pfau@pfau@sedielem @koraykv @MarcRanzato @rob_fergus @ylecun Huh, wonder what those authors are up to these days.
#4Yann LeCun@YLECUN@sedielem @koraykv @MarcRanzato @rob_fergus Back when we thought deep net pre-training could be done with stacked auto-encoders trained with group sparsity regularization.
Sander Dieleman@sedielemThis one blew my mind back in the day! From "Learning Invariant Features through Topographic Filter Maps" (2009) by @koraykv, @MarcRanzato, @rob_fergus and @ylecun
#4Yann LeCun@YLECUN@sedielem @koraykv @MarcRanzato @rob_fergus It does work. I mean, with stacks of pre-trained sparse convolutional auto-encoders, we could get to a good starting point from which to fine-tune supervised on tiny labelled datasets (like Caltech 101, that had 30 training samples per category) and get near-SOTA performance.
Sander Dieleman@sedielem@ylecun @koraykv @MarcRanzato @rob_fergus I mean... it would probably still work, just a bit tedious to do it 100 times 😂
#75Sander Dieleman@SEDIELEMThis one blew my mind back in the day! From "Learning Invariant Features through Topographic Filter Maps" (2009) by @koraykv, @MarcRanzato, @rob_fergus and @ylecun
Roland Memisevic@RolandMemisevicRelics from the prehistoric era of AI
@ylecun @koraykv @MarcRanzato @rob_fergus I mean... it would probably still work, just a bit tedious to do it 100 times 😂
Yann LeCun@ylecun@sedielem @koraykv @MarcRanzato @rob_fergus Back when we thought deep net pre-training could be done with stacked auto-encoders trained with group sparsity regularization.
#121Taco Cohen@TACOCOHENI remember the days when a successful experiment was one that produced cool filters
Sander Dieleman@sedielemThis one blew my mind back in the day! From "Learning Invariant Features through Topographic Filter Maps" (2009) by @koraykv, @MarcRanzato, @rob_fergus and @ylecun
#143David Pfau@PFAU@sedielem @koraykv @MarcRanzato @rob_fergus @ylecun Huh, wonder what those authors are up to these days.
Sander Dieleman@sedielemThis one blew my mind back in the day! From "Learning Invariant Features through Topographic Filter Maps" (2009) by @koraykv, @MarcRanzato, @rob_fergus and @ylecun
#173Saining Xie@SAININGXIEon the topic of favorite filters…
first emergence, mostly accidental
Taco Cohen@TacoCohenI remember the days when a successful experiment was one that produced cool filters
@sedielem @koraykv @MarcRanzato @rob_fergus @ylecun
Sander Dieleman@sedielemThis one blew my mind back in the day! From "Learning Invariant Features through Topographic Filter Maps" (2009) by @koraykv, @MarcRanzato, @rob_fergus and @ylecun
@sedielem @koraykv @MarcRanzato @rob_fergus @ylecun Good old days! These words were also very influential in neuroscience. We kinda lost that direct connection
Sander Dieleman@sedielemThis one blew my mind back in the day! From "Learning Invariant Features through Topographic Filter Maps" (2009) by @koraykv, @MarcRanzato, @rob_fergus and @ylecun