Every time I give out alpha (like linalg fixes for shampoo), someone is worried that I am accelerating the labs who could send their models to read my tweets and implement. But then I realize we are here to improve the human condition with better technology and we should make friends along the way, even if you make them mad for a little bit.
CoreAutoAI's Rohan Anil defends sharing training optimizations like Shampoo fixes, arguing collaboration beats proprietary secrecy
Critics worry sharing optimizations lets rivals train models faster.
Most Activity
this is the right mindset!
Every time I give out alpha (like linalg fixes for shampoo), someone is worried that I am accelerating the labs who could send their models to read my tweets and implement. But then I realize we are here to improve the human condition with better technology and we should make friends along the way, even if you make them mad for a little bit.