AI developer Teortaxes argues the distinction between genuine intelligence and semantic processing in LLMs is functionally irrelevant

@teortaxesTex It matters because the current LLM architecture won't scale to AGI, at the very least not in any way that is economically achievable, let alone ASI. Any company that rushes there would just go bankrupt trying to build sufficient infra and push back AI again

@invizive @Donogzs delusional we're not even close to maxing it out

what does it matter how you call them?

@teortaxesTex @Donogzs Delusional Having to go through all the context for each inference is a fundamental flaw of this architecture that is a doom for scaling

The only reasonable expectation of hitting singularity with it was that we'd find enough crutches until it invents a better architecture

@teortaxesTex Aren't you the creative type. Have a beautiful day in your dreamland!

@Donogzs @teortaxesTex "All you need is attention" was not a mistake, but it exhausted itself for a reasonable return half a year ago at least

The architecture is fundamentally flawed, we can't even be sure that current cores would be beneficial for the next one

@teortaxesTex They certainly can make cybersec situation worse, to a point of being on par with strategic weapons

But it is nowhere near the initial value proposition of a "digital deity" and permanent overclass that attracted major investments in the first place, so it is a disappointment

@invizive @Donogzs Yet you will keep seeing labs turn profitable, models become more capable, at a *faster* pace if anything, retard. I don't have patience for this mush anymore. muh fundamental flaw we've slashed costs of attention by OOMs now

@teortaxesTex The gold rush is overextended, we've hit no new emergent abilities for a while now - even Fable was just able to oneshot larger prototypes, nothing groundbreaking

If current LLM craze ends up making investors regret it the whole field will be set back by a decade

@invizive @Donogzs what does it even matter if true (it's not) infra for LLMs is generic infra for DL, unless you have some strong argument against parallelism Suppose LLMs stall in 2028 at some Fable 3 level. So what, we just focus on non-LLM research bears just make empty noises now

@invizive oh oh only larger prototypes how large does it have to be to fuck up your infrastructure?

@Donogzs @teortaxesTex I had little time compiling info, but I'd coincidentally go for mid-2028 at the latest, with the curve already being hit now

Using projection of hitting ~1x YoY improvement on METR 80% TH as the guessing point

@invizive sorry that reality is disappointing

@teortaxesTex @Donogzs "Slashing by OOMs" means nothing to geometric growth in demand

They aren't getting better at faster pace than three years ago, new emergent properties don't occur anymore

@invizive @teortaxesTex When will you guys realize that architectural changes is part of "scale"? Or are you claiming we need to start over from the drawing board? And how far back if so?

@invizive @teortaxesTex Then it should be pretty trivial to fit an S curve and work out the exact date of plateau from first principles no? Give us a number then. Otherwise this is unfalsifiable larp

@teortaxesTex This meme is 10/10. Thanks for sharing this

@_Eddited_ @teortaxesTex Died 1453, born 2026, welcome back Constantine XI!

@invizive @Donogzs the whole "emergent properties" framing was confused if the demand grows geometrically, this is argument about their success unto itself anyway, we'll have continual learning with LLMs by 2028

@teortaxesTex @Donogzs IDGAF about economics, it poisons the AI research at this point - the subject I was interested in since 2014

All I care about in this regard is actual development

And current tensor cores are optimized for very specific low-precision matrix operations, not for all parallelism