Rohan Anil, formerly a Distinguished Engineer at Google DeepMind, says Ant required no permissions or meetings to obtain compute resources, enabling uninterrupted fundamental research
Replies recommend low-interest topics and language diffusion for modest hardware work.
One thing about being scrappy is that I just cannot get 20 petabytes of disk overnight so I can train a better model, but the fact that I don’t have 20 petabytes immediately got me thinking about a clever way to do the same think by trading compute for it.
It’s age of research, but we need to advance research on faster timelines. Org chart and status driven tastes on research is not going to work. Its needs to be cradled like a baby, need to pay attention, need to do diaper changes and feed that research.
rohan anil@_arohan_Saying this in the most helpful way: since I am not cherrypicking anything here, Interesting bit having worked at Ant and GDM. At Ant, I needed no permissions or meetings to get access to them. No trading compute, no future promises. No stress. Just pure research, and it helped a lot in landing new innovative ideas to make model better and it was one of the most beautiful time of my life for pushing fundamental innovation, my mind was fully refreshed and became the best IC I have ever been, even better than when I was at Brain. Only Jt could nerdsnipe me into something new because I think its possible to do that soon.
Cortisol spike on all or nothing compute fights is something anti thesis to fundamental innovation.
rohan anil@_arohan_It’s age of research, but we need to advance research on faster timelines. Org chart and status driven tastes on research is not going to work. Its needs to be cradled like a baby, need to pay attention, need to do diaper changes and feed that research.
#150David Pfau@PFAU@_arohan_ My solution to this is to work on stuff no one cares about so no one has tried to scale it yet, so I can still do novel work with like 5 GPUs.
rohan anil@_arohan_Saying this in the most helpful way: since I am not cherrypicking anything here, Interesting bit having worked at Ant and GDM. At Ant, I needed no permissions or meetings to get access to them. No trading compute, no future promises. No stress. Just pure research, and it helped a lot in landing new innovative ideas to make model better and it was one of the most beautiful time of my life for pushing fundamental innovation, my mind was fully refreshed and became the best IC I have ever been, even better than when I was at Brain. Only Jt could nerdsnipe me into something new because I think its possible to do that soon.
#150David Pfau@PFAU@_arohan_ It's a quiet life but it has its simple joys.
David Pfau@pfau@_arohan_ My solution to this is to work on stuff no one cares about so no one has tried to scale it yet, so I can still do novel work with like 5 GPUs.
#150David Pfau@PFAU@LucaAmb @_arohan_ Oh no no, I'm talking even fewer people caring.
Luca Ambrogioni@LucaAmb@pfau @_arohan_ Join the language diffusion field then :D
#488kache@YACINEMTB@_arohan_ A lot of foundational research doesn't need that much compute..!
rohan anil@_arohan_It’s age of research, but we need to advance research on faster timelines. Org chart and status driven tastes on research is not going to work. Its needs to be cradled like a baby, need to pay attention, need to do diaper changes and feed that research.
@_arohan_ the closer you are to zero sum environments the more politics creeps in; escape zero sum, it's the only way
rohan anil@_arohan_Saying this in the most helpful way: since I am not cherrypicking anything here, Interesting bit having worked at Ant and GDM. At Ant, I needed no permissions or meetings to get access to them. No trading compute, no future promises. No stress. Just pure research, and it helped a lot in landing new innovative ideas to make model better and it was one of the most beautiful time of my life for pushing fundamental innovation, my mind was fully refreshed and became the best IC I have ever been, even better than when I was at Brain. Only Jt could nerdsnipe me into something new because I think its possible to do that soon.
@pfau @_arohan_ Join the language diffusion field then :D
David Pfau@pfau@_arohan_ My solution to this is to work on stuff no one cares about so no one has tried to scale it yet, so I can still do novel work with like 5 GPUs.