Charles Frye proposed training machine learning models with large amounts of implementation noise to increase robustness against floating-point nondeterminism and nonassociativity

B(Ben (no treats)@andersonbcdefg

@charles_irl ITS JUST REGULARIZATION

C🎉Charles 🎉 Frye @ MLSys@charles_irl

my gut says that to solve float numerics problems from nondeterminism x nonassociativity, we need to think bigger than determinism. models could eg be trained with large amounts of "implementation noise" so that the learned network is more robust to implementation skew.