Gary Marcus argues massive AI infrastructure investments are flawed because generative AI lacks competitive moats

@TheStalwart @GaryMarcus

Em um mercado repleto de euforia e irracionalidade, se deparar com um comentário lógico e racional é como achar um oasis no deserto.

any day now. just you wait

@RomeoLupascu they are subsumed in part under number 2; they part of the everybody building the same thing

@GaryMarcus for the model, no, you're right.

But if youre google and you have an ads + search based ecosystem (and cloud), you have a good business case to invest in what is massive demand gen for those revenue streams. You'd think ant/oAI would try to backward integrate to escape.

This is I how see things as well. The only explanation that makes sense is that decision-makers in many companies believe that inference demand will be so astronomical, and price sensitivity so low, that each and every infrastructure investment will pay off.

@jedpolglase you seem to have skipped #4?

@GaryMarcus No sir…you are not.

i wonder if/when we reach the upper part of the sigmoid for this class of models, the research costs drop and the profits increase. also as enterprises pick a default AI cloud, the opportunity to shift share diminishes & subsidies will likely diminish.

also the per token pricing increases in just the last quarter or two point to better pricing power. https://tomtunguz.com/ai-model-inflation/

ec2 & s3 cut prices more than 100x in the first few years, model companies are doing the opposite. like you say, anthropic generating a profit...

@GaryMarcus America will shortly get yet another reminder of why capitalism requires communal supervision and control.

Thanks everyone for reading! Longer version some additional points on crowd psychology and links to a pair of great new podcasts with the legendary investors @realsteveeisman and @gnoble79 and converging words from @BradDeLong here: https://open.substack.com/pub/garymarcus/p/why-things-will-eventually-fall-apart?r=8tdk6&utm_campaign=post-expanded-share&utm_medium=web

@GaryMarcus Recursive self-improvement and exponential take-off thanks to algorithmic gains favor a winner take all scenario (even with limited compute)? 🙃

@GaryMarcus "Essentially the same technical solution with essentially the same data" is wrong. Having no temptation to regard these programs as sentient, I find Gemini better at one kind of question, Claude at another. Useful in both cases, but I wouldn't pay for either.

You only get commodity pricing when there’s enough supply to meet demand and we’re not there yet on the infra side.

Inference costs have come down but supply is still constrained

Google’s bet is on vertical integration and ecosystem ownership centred around their custom TPUs, which is very different from pure non-differentiated competition

I don’t disagree with your points more broadly. Things are trending towards commoditisation as model capabilities converge. But in Google’s case the reasons for investment seem sound:

> lower internal cost of compute > rapid payback at premium prices before the supply/demand curve flattens > ecosystem lock-in for long term revenue security

@GaryMarcus The bet is that the prompts and inputs themselves are the data (not a controversial idea for, say, self-drive), in which case there is a winner takes all network effect.

@ThinkFinance999 @GaryMarcus TPUs don’t need to be magic. Switching costs just need to be high enough to dissuade labs from cancelling contracts and moving to other infrastructure. Regardless, Google benefits from reduced internal cost of compute and can serve their models cheaper.

@GaryMarcus Probably a lot, unless you’re sitting in the rooms where the decisions are being made. I think assuming all these companies are completely ignorant is misguided. Some may be, but do you think infra investments like this one won’t pay off? They don’t have to win “AI” to make $

@GaryMarcus These companies are still thinking in the "social media" mindset, but there is no more network effect. You can switch AI easily, and most people do within the same task.

Ironically, AI will kill many of the current tech giants.

@GaryMarcus Couldn't you make the same argument about the compute market and the storage market in the cloud, eg ec2 and s3?

AWS generates more than 30% in operating income in a commodity business. What's different with GPUs?

@GaryMarcus This is how I see it - Converging models -> AI becomes commodity -> AI will be undifferentiated utility (like internet, power, water) -> Data centers are token factories but price of tokens is going to zero

Massive AI data center investments/debt -> Big losses