AI Judge changed title after evaluation, original title: "Yuchen Jin, who builds AI systems and products at Databricks, says H100 GPU shortages have intensified with prices now higher than three years ago and no on-demand supply available as major labs lock up capacity through multi-year contracts"
Developers increasingly rely on older A100 GPUs amid constraints.
GPU shortage is worse than ever.
H100s cost more today than they did 3 years ago, and you cannot get them on-demand.
The big AI labs have locked up most of the supply for years. I’m worried university researchers and individual developers simply won’t be able to get GPUs.
Users are upset that large AI labs are locking up H100 supply and worsening the GPU shortage for researchers by driving up prices, blocking access, and stifling innovation, while some remain optimistic that scarcity will spur smaller teams.
No Digg Deeper questions have been answered for this story yet.
the gpu crunch is real
GPU shortage is worse than ever.
H100s cost more today than they did 3 years ago, and you cannot get them on-demand.
The big AI labs have locked up most of the supply for years. I’m worried university researchers and individual developers simply won’t be able to get GPUs.
So much for technological obsolescence of GPUs.
GPU shortage is worse than ever.
H100s cost more today than they did 3 years ago, and you cannot get them on-demand.
The big AI labs have locked up most of the supply for years. I’m worried university researchers and individual developers simply won’t be able to get GPUs.
currently using A100s like an animal
GPU shortage is worse than ever.
H100s cost more today than they did 3 years ago, and you cannot get them on-demand.
The big AI labs have locked up most of the supply for years. I’m worried university researchers and individual developers simply won’t be able to get GPUs.
H100 is more expensive now than it was when released, ,is a 3 y/o hardware, and you can't find any available on demand!
Inference is everything.
The stock market has no idea
GPU shortage is worse than ever.
H100s cost more today than they did 3 years ago, and you cannot get them on-demand.
The big AI labs have locked up most of the supply for years. I’m worried university researchers and individual developers simply won’t be able to get GPUs.
@realsigridjin have to at @TheAhmadOsman here
@Yuchenj_UW lambda has h100s on demand avail rn, just checked. But yeah over twice the cost that they were a couple years ago, and often hard to snag. You can write a script to use their api to snag em though when times are tough.
@Yuchenj_UW @0xaiwa http://Admin.hotaisle.app
@Yuchenj_UW i hate this! reminds me of 2023 and it was really hard to get anything done with no GPUs!
@Yuchenj_UW buying gpus not cost but asset
@Yuchenj_UW been saying this for a while but nobody listens till their model cant train
@Yuchenj_UW @jun_song I think we will have more breakthrough in terms of architecture and smaller models in the future will be as good as mythos. Meta and others are building data centres and at some point they will unlock them for consumer use as demand for giant GPU clusters will decrease
Why the hell isn't anybody using digital neuromorphics.
They're way cheaper. They use virtually no power comparatively, produce almost no waste heat because of that.
They're way easier to produce. We could make them all here with a lot less 'high tech' equipment. And, they scale infinitely.
Now to be fair, my previous startup's plans for these was to just stack NM500 chips to scale three levels of basic AI for robots to do fairly simple things.
But, the scales we were looking at were also small. If somebody stacked these like they do right now with GPUs in these giant data centers it could likely rival those.
Issue is that all the 'programming' would have to basically start from scratch because neuromorphics learn, and can then be locked in. They can also however be copied, that is their synaptic weight spaces are directly transferable to others of the same make and model. Porting to other models would require a slightly complex reconfiguring algo.
@Yuchenj_UW W @jino_rohit
@Yuchenj_UW AMD R9700.
The compute bottleneck is real and it risks concentrating AI capability development in just a few labs. Innovative paths like better model efficiency, new hardware paradigms or shared secure compute clusters could open this up while maintaining strong security standards. We can't let access become the new moat.
@Yuchenj_UW i've had a bunch of private individuals pool their home gpu's, people like @TheAhmadOsman abroad supply their home gpu's Ghola so organizations like universities can train
@Yuchenj_UW this is why Lightning AI, the folks behind PyTorch Lightning, have a multi-cloud offering w seamless switching https://lightning.ai/pricing
@xlr8harder i’m a fucking barbarian then (v100)
@Yuchenj_UW So basically they are controling supply of GPU that means their controlling the price and access and now we have to buy from them for tokens and cannot use open source
@Yuchenj_UW Health Science must have access. I seriously don't give a fuck about a new feature on some Social App.
I do care about advancements that are made towards solving chronic illness. Like cancer!