Leading AI companies intend to recruit software engineers and AI researchers to build systems automating software engineering and AI research at over 100 times current rates toward superintelligent AI

@AISloppyJoel @peterwildeford Super intelligent AI systems will, by definition, not be under our control, we won't be the apex species anymore. So, our future is uncertain but it's unlikely we win. Do we consider ant hills when building a highway? No. Predictions include gradual disempowerment and extinction.

100x faster is an extreme level of acceleration that we haven't come close to seeing in the real world yet. I'm not complaining if it happens! But at some point I think we have to look around and ask what the actually-existing version of the technology is capable of achieving, not ascribe unlimited abilities to a future version of it.

@AISloppyJoel It may not be up to us to decide

@peterwildeford Imagine trillions of instances of models that can via RLVR advances solve Erdos problems, Riemann hypothesis, kernel optimizations, new computer languages/compilers from the hip (w/ zero inference time). Hard to know the knock on effects.

@peterwildeford Spot on. It's actually hard to fathom how weird the world could soon be (even in the best cases)

Fortunately there’s probably at least 2-3 more years where I’d bet on slow takeoff, then after that you could have foom and wake up one morning with Sam Altman as Emperor (or worse). IMO that’s the window to enact meaningful policy, safety guardrails, coordinate, alignment research.

@peterwildeford What's your source for saying software engineers aren't being hired anymore? 🤔

@_everythingism what I meant to be said is that "at these top AI companies, the top 0.01% engineers get hired but the top 1% engineers who used to get hired no longer get hired"

@_everythingism I didn't say that software engineers aren't being hired anymore. I said that within Anthropic, Google, and OpenAI the hiring is changing in big ways.

@_everythingism I didn't say it would be some linear, predictable process. I do think there's a lot of uncertainty in how this plays out, including that it just doesn't work at all. But I do think it's plausible AI really accelerates what is a non-linear process.

@peterwildeford But is the software development on the critical path? I'd think we're probably at the point where it's the months-long training runs that are the bottleneck.

@peterwildeford OK sure, it's changing. But you said "even these top 1% engineers don't get hired anymore" and seemed to be suggesting this is a measure of AI progress.

Is there some metric you're using to say that? AI companies have said they plan to massively expand their workforce this year (doubling the number of people in the case of OpenAI). Maybe they can do that and only hire top 0.01% engineers but I'm not sure how that's even being defined.

@_everythingism We've made a bunch of progress in automating stuff. What happens if we automate the ability to make progress in automating stuff and we progress 100x faster?

@peterwildeford >it's very much on track.

Is it?

>AI writes the vast majority of the code instead of humans.

Full of security bugs and unexplainable internals, ambiguous correctness verified by incomplete test suites.

@_everythingism Engineering hiring comes in tiers, you can look at the difference between L1-L4 engineer hiring and L5+ engineer hiring.

@smchatter1 > Is it?

Yes

> Full of security bugs and unexplainable internals, ambiguous correctness verified by incomplete test suites.

The AI-generated code works quite well! Of course, it's not perfect, but neither is human-generated code

@peterwildeford Don't get be wrong, I'm sure it's still very competitive to get a job at these companies. But you were making it sound like AI has raised the bar to a much higher level than it would otherwise be.

@peterwildeford OK...but looking at the job listings on the websites of these companies, it does not look to me like they are only "top 0.01%."

@_everythingism I am saying that AI has raised the bar to a much higher level than it would otherwise be