Positive users praise Jensen's strategic foresight in acquiring Groq after the Cerebras deal to secure future inference tech, while negative users reject the panic-buy framing as ridiculous and overstated.
Based on 12 visible X reactions from 67 accounts; directional sample.
Ask a question below.
Published answers will appear here.
@jukan05 Jensen playing 5D chess while everyone else is playing checkers. The "Silicon FBI" angle is wild but honestly tracks with how these mega-deals go down 🎯 NVIDIA buying reactively is peak competitive paranoia energy
@jukan05 Groq kills inference margins. jensen didn't panic-buy groq, he bought the future before it became obvious. that's the expensive part. everything else is noise.
@jukan05 you retards act like your more knowledgable than a ceo of the largest company of all time
@Tim_Dettmers that's a wild take on the architecture, honestly makes so much sense.
The strategy aims to lower LLM inference costs alongside upcoming Rubin systems.
@jukan05 Dead wrong. Both of the rumors and your thoughts are just completely off.
I was surprised too by the Groq acquisition, but when seeing Vera Rubin it all made sense: MoE layers in decode are a severe bottleneck and heavily bandwidth bound. With strong networking, layers can be distributed effectively over SRAM. SRAM devices are also very cheap to manufacture, so this can be a big win. This is similar to using GDDR7 for (partial) prefill -- cheaper to manufacture while yielding the better cost/performance due to the FLOPS-bound nature of prefill. You still need HBM for attention layers in decode, though. The picture is clear. You want strong hardware-inference co-development and Groq is the right move for MoE layers in decode. Overall, Cerebras is quite a different beast from Groq chips due to its size and would be used beyond MoE layers in decode. You basically want to do both attention decode layers and MoE decode layers with Cerebras. But the problem is, you need extremely high-speed interconnects for this, or otherwise you run into large network bottlenecks. I know that there are non-public interconnects that can do this, but not sure how many datacenters OpenAI has with these interconnect systems. There are other hardware + architecture + algorithm co-development. We are already seeing the landscape changing. My lab also has something in the pipeline that could shift the hardware co-design angle. This entire subfield is highly agile and is changing rapidly.
Positive users praise Jensen's strategic foresight in acquiring Groq after the Cerebras deal to secure future inference tech, while negative users reject the panic-buy framing as ridiculous and overstated.
Based on 12 visible X reactions from 67 accounts; directional sample.
Ask a question below.
Published answers will appear here.
I was surprised too by the Groq acquisition, but when seeing Vera Rubin it all made sense: MoE layers in decode are a severe bottleneck and heavily bandwidth bound. With strong networking, layers can be distributed effectively over SRAM. SRAM devices are also very cheap to manufacture, so this can be a big win. This is similar to using GDDR7 for (partial) prefill -- cheaper to manufacture while yielding the better cost/performance due to the FLOPS-bound nature of prefill. You still need HBM for attention layers in decode, though. The picture is clear. You want strong hardware-inference co-development and Groq is the right move for MoE layers in decode. Overall, Cerebras is quite a different beast from Groq chips due to its size and would be used beyond MoE layers in decode. You basically want to do both attention decode layers and MoE decode layers with Cerebras. But the problem is, you need extremely high-speed interconnects for this, or otherwise you run into large network bottlenecks. I know that there are non-public interconnects that can do this, but not sure how many datacenters OpenAI has with these interconnect systems. There are other hardware + architecture + algorithm co-development. We are already seeing the landscape changing. My lab also has something in the pipeline that could shift the hardware co-design angle. This entire subfield is highly agile and is changing rapidly.