Rohan Anil, formerly a Distinguished Engineer at Google DeepMind, says Ant required no permissions or meetings to obtain compute resources, enabling uninterrupted fundamental research

It’s age of research, but we need to advance research on faster timelines. Org chart and status driven tastes on research is not going to work. Its needs to be cradled like a baby, need to pay attention, need to do diaper changes and feed that research.

Cortisol spike on all or nothing compute fights is something anti thesis to fundamental innovation.

@_arohan_ My solution to this is to work on stuff no one cares about so no one has tried to scale it yet, so I can still do novel work with like 5 GPUs.

One thing about being scrappy is that I just cannot get 20 petabytes of disk overnight so I can train a better model, but the fact that I don’t have 20 petabytes immediately got me thinking about a clever way to do the same think by trading compute for it.

@_arohan_ It's a quiet life but it has its simple joys.

@_arohan_ the closer you are to zero sum environments the more politics creeps in; escape zero sum, it's the only way

@Viby_Matty I just talked about my Ant experience. For google experience, you need to read bloomberg.

@_arohan_ How much of the gap was political (org permission) vs structural (compute scarcity at GDM) in your experience ?

@pfau We love you for that!

@pfau @_arohan_ Join the language diffusion field then :D

I largely post on twitter, mainly as way of of soaking in entropy - there is lot of focus and craft going in the background, it just happens that our minds are too awake and need constant entropy to not collapse.

@LucaAmb @_arohan_ Oh no no, I'm talking even fewer people caring.

@_arohan_ A lot of foundational research doesn't need that much compute..!

@_arohan_ this is $50k/mo if you buy the same sku i have at hetzner x100

That sounds amazing. The Dream!! What i heard, the word, is AUTONOMY - something that probably aligns with a company with most those letters. Honestly though doing it out on your own, in a dark room with everything you built yourself... I could never trade what I built for corporate environment. Everyday a montage to greatness. Its all a dream innit https://music.youtube.com/watch?v=Jx53g34xAyE&si=LjnL-MW67kAR-UdR

@_arohan_ I think this is a very important point. Low-friction access to compute is how you get fundamental innovation, faster iteration, and better model ideas.

@_arohan_ isn't it obvious. Ant is a lab and google is a product company. Sure, Ant is building an API business, but it is still fundamentally a research lab. Google is self funded and their products need to remain competitive to generate revenue, which fund the lab.

@_arohan_ mighty interested in that something new

I remember brain times when all the theory researchers donated their credit to me so I can make theory into practice. Such a good time and memory!

@_arohan_ Google is a 180,000 people organization vs Ant