—
Not ranked yet
—
Not ranked yet
@MLStreetTalkRT @sarahookr: On the slow death of scaling.
@BlackHC@sarahookr I thought the slow death of scaling was about compute scaling. Isn't the claim here about algorithm improvements vs data improvements while compute remains more or less fixed? 🙃
@sarahookr@BlackHC Actually the main point is now that transformer innovation is saturated it doesn’t make sense to continue to scale model size. So very much in keeping with saying that innovation in architecture is not yielding returns.@MLStreetTalkRT @sarahookr: On the slow death of scaling.@BlackHC@sarahookr I thought the slow death of scaling was about compute scaling. Isn't the claim here about algorithm improvements vs data improvements while compute remains more or less fixed? 🙃@sarahookr@BlackHC Actually the main point is now that transformer innovation is saturated it doesn’t make sense to continue to scale model size. So very much in keeping with saying that innovation in architecture is not yielding returns.@MLStreetTalkRT @sarahookr: On the slow death of scaling.@BlackHC@sarahookr I thought the slow death of scaling was about compute scaling. Isn't the claim here about algorithm improvements vs data improvements while compute remains more or less fixed? 🙃@sarahookr@BlackHC Actually the main point is now that transformer innovation is saturated it doesn’t make sense to continue to scale model size. So very much in keeping with saying that innovation in architecture is not yielding returns.@MLStreetTalkRT @sarahookr: On the slow death of scaling.@BlackHC@sarahookr I thought the slow death of scaling was about compute scaling. Isn't the claim here about algorithm improvements vs data improvements while compute remains more or less fixed? 🙃@sarahookr@BlackHC Actually the main point is now that transformer innovation is saturated it doesn’t make sense to continue to scale model size. So very much in keeping with saying that innovation in architecture is not yielding returns.
Comments