It details how Adam succeeds empirically despite flawed convergence proofs
I am starting a blog about deep learning theory and its value to practitioners! First post is about Adam, broken convergence proofs, and what theory can contribute when stuff just works anyways without it. Subscribe on Substack if you like it! https://undertheassumptions.substack.com/p/the-optimizer-that-outlived-its-proof
No Digg Deeper questions have been answered for this story yet.
As an empirical researcher, I often find intuitions from ML theories cool but the assumptions confusing. This is a great blog series to help practitioners understand ML theories and when they can be useful!
I am starting a blog about deep learning theory and its value to practitioners! First post is about Adam, broken convergence proofs, and what theory can contribute when stuff just works anyways without it. Subscribe on Substack if you like it! https://undertheassumptions.substack.com/p/the-optimizer-that-outlived-its-proof
I recommend giving this post a read for some theory x practice + philosophy of science!
and excited to see what comes next for the blog👀
I am starting a blog about deep learning theory and its value to practitioners! First post is about Adam, broken convergence proofs, and what theory can contribute when stuff just works anyways without it. Subscribe on Substack if you like it! https://undertheassumptions.substack.com/p/the-optimizer-that-outlived-its-proof
