Deep Learning Generalization Stems From Network Geometry And Optimization

@teortaxesTex i am lowkey convinced that "sample efficiency" as a phrase has sort of hurt the discourse on generalization as a phenomenon by modeling it as a learning speed derivative it skips asking "how do we make *sharing* representations the default"

@teortaxesTex "dog circuit overlaps with pet circuit which is also shared with the cat circuit" - this emerging at all from local gradient steps makes future steps *compound* by default. afaic it's not super understood as to how to make this happen more often on purpose at the optimizer level!