• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
AI
Report

Power and manufacturing bottlenecks in the AI chip buildout

a16z says former Intel CEO Pat Gelsinger discussed why faster chip design alone won’t solve manufacturing and memory bottlenecks.

Garry TanGT
a16zA1
ElenaEL
3 Sources, 1h ago, first seen 1h ago

TLDR

a16z says its discussion with former Intel CEO Pat Gelsinger examined how AI could transform chip design, even as manufacturing, memory bandwidth, advanced packaging and power remain constraints. Gelsinger argues that U.S. energy capacity was essentially flat for 10 to 15 years and warns that data centers need enough power to run their GPUs. The discussion also considers whether specialized AI chips might consolidate around fewer architectures.

Combined views

9.6K

3 Sources, first seen 1h ago

32 likes11 comments14 saves8 reposts

Combined views

9.6K

3 Sources, first seen 1h ago

32 likes11 comments14 saves8 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Featured Source

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

3 Sources

a16z@a16z"Energy Capacity = Economic Capacity" Former Intel CEO Pat Gelsinger on why America's power grid is the ceiling on growth: "We went through 10 to 15 years where essentially I was taking coal offline at the rate I was adding renewables, and essentially the nation was flatlined in terms of overall energy capacity." "So essentially our economic capacity as a nation, flat for fifteen years." "Even with that, we've gone to maybe increasing our national energy capacity 4% per year. Going from essentially zero to one." "Nuclear as base load is a fabulous thing that we want to go build more of. Unfortunately all of our renewables have deep dependencies in China." "Energy capacity somewhat dampens how euphoric we can get on AI. Why build a new data center and buy the million GPUs if I can't power them?" @PGelsinger @RaghuRaghuram @appenz1h
Elena@VirtualElenapat gelsinger (@PGelsinger) designed chips at intel in a period when doing so could involve writing your own hardware description language, which he did for the 486. his team also built a compiler and tools for automating placement and routing. “I wrote my own language,” he says in this pod with @RaghuRaghuram and @appenz. it's a good reminder of how many things had to be invented alongside the processors themselves, and why the prospect of AI taking over more of that work is so exciting. pat then asks us to imagine designing an excellent AI accelerator in three months. getting it fabricated, packaged, and into a working rack takes roughly nine more in his example (the relevant unit is now the rack: “nothing's a chip anymore, it's a rack”). so there's a period in which your very clever design has to sit and wait for its physical existence, while the models whose workloads you've designed it for can keep evolving. his description of graphcore is that it “wasn't a bad design, but the world moved on.” i think about how often people announce a new model or a new way of using one, and nine months seems like a very long commitment to your current understanding of what the hardware ought to do. guido's argument is that agents should make it easier to support a greater variety of architectures, because writing all the software to use an unfamiliar chip has historically been one of the barriers. pat's answer is that somebody still has to pay for manufacturing and for a place to put all these chips. he expects consolidation, even with better software tools. if you're a customer committing to a data center and its electricity supply, you're making a rather different bet than the engineer who's just got a new design to work. which leaves a lot for hardware people to do. pat is especially dissatisfied with hbm (high-bandwidth memory), pointing to problems with density, bandwidth, and heat; he calls it “a hideous memory,” and says he's just funded a new memory company that's still in stealth. he wants better memory closer to compute, optical connections between systems, and less electricity lost in conversion on its way to the chip. he also seems very happy to talk about cooling. “engineers are becoming plumbers,” is how he puts it. if we're excited about being able to design more things with AI, we should be excited about the work that allows us to manufacture and operate them, too. pat has already lived through a period of having to invent quite a lot of the surrounding machinery. it makes sense that he'd see another one as an opportunity.22m
Garry Tan@garrytanRT @VirtualElena: pat gelsinger (@PGelsinger) designed chips at intel in a period when doing so could involve writing your own hardware des…21m
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI

    3 Sources

    a16z@a16z"Energy Capacity = Economic Capacity" Former Intel CEO Pat Gelsinger on why America's power grid is the ceiling on growth: "We went through 10 to 15 years where essentially I was taking coal offline at the rate I was adding renewables, and essentially the nation was flatlined in terms of overall energy capacity." "So essentially our economic capacity as a nation, flat for fifteen years." "Even with that, we've gone to maybe increasing our national energy capacity 4% per year. Going from essentially zero to one." "Nuclear as base load is a fabulous thing that we want to go build more of. Unfortunately all of our renewables have deep dependencies in China." "Energy capacity somewhat dampens how euphoric we can get on AI. Why build a new data center and buy the million GPUs if I can't power them?" @PGelsinger @RaghuRaghuram @appenz1h
    Elena@VirtualElenapat gelsinger (@PGelsinger) designed chips at intel in a period when doing so could involve writing your own hardware description language, which he did for the 486. his team also built a compiler and tools for automating placement and routing. “I wrote my own language,” he says in this pod with @RaghuRaghuram and @appenz. it's a good reminder of how many things had to be invented alongside the processors themselves, and why the prospect of AI taking over more of that work is so exciting. pat then asks us to imagine designing an excellent AI accelerator in three months. getting it fabricated, packaged, and into a working rack takes roughly nine more in his example (the relevant unit is now the rack: “nothing's a chip anymore, it's a rack”). so there's a period in which your very clever design has to sit and wait for its physical existence, while the models whose workloads you've designed it for can keep evolving. his description of graphcore is that it “wasn't a bad design, but the world moved on.” i think about how often people announce a new model or a new way of using one, and nine months seems like a very long commitment to your current understanding of what the hardware ought to do. guido's argument is that agents should make it easier to support a greater variety of architectures, because writing all the software to use an unfamiliar chip has historically been one of the barriers. pat's answer is that somebody still has to pay for manufacturing and for a place to put all these chips. he expects consolidation, even with better software tools. if you're a customer committing to a data center and its electricity supply, you're making a rather different bet than the engineer who's just got a new design to work. which leaves a lot for hardware people to do. pat is especially dissatisfied with hbm (high-bandwidth memory), pointing to problems with density, bandwidth, and heat; he calls it “a hideous memory,” and says he's just funded a new memory company that's still in stealth. he wants better memory closer to compute, optical connections between systems, and less electricity lost in conversion on its way to the chip. he also seems very happy to talk about cooling. “engineers are becoming plumbers,” is how he puts it. if we're excited about being able to design more things with AI, we should be excited about the work that allows us to manufacture and operate them, too. pat has already lived through a period of having to invent quite a lot of the surrounding machinery. it makes sense that he'd see another one as an opportunity.22m
    Garry Tan@garrytanRT @VirtualElena: pat gelsinger (@PGelsinger) designed chips at intel in a period when doing so could involve writing your own hardware des…21m
    Today's Rank

    #9

    Today's Rank

    #9