Jennifer finds changing her name to Jeff on an AI resume tool triggers more limiting career feedback

trillion dollar sexism tool

@ramez I wonder how much of this is due to pre-training and how much to RLHF

@LionBlogosphere @GaryMarcus Right, which is why we shouldn’t allow the societal bias tool to do shit like rate job candidates.

@GaryMarcus AI is misaligned because it's not able to differentiate itself. That's a big issue for psychology.

https://abowenkc.substack.com/p/the-devolution-of-society-differentiation?r=72tmbx

@GaryMarcus It's probably just connecting the latent space for female names and real female written resumes.

@GaryMarcus men are not women and women are not men; however, men and women are human.

... not sure what any of this proves, other than men and women function differently by and large.

@BrunoCuzzi I would guess that RLHF moderates these biases, and most comes from pre-training

@GaryMarcus AI doesn’t ‘understand’ anything, it’s autocomplete on steroids, hallucinating the most stats likely version of you from whatever garbage it scraped off the internet. That’s the mirage: a coherent-sounding persona dragging every stereotype in its training data along for the ride.

@GaryMarcus Sexism is human, LLMs just predict the next word based on the human patterns in its training data.

@ramez Saw a video of a robot unlocking a padlock with a key. It shuffled the key back and forth between claws to orient the key for insertion. Even though it had oriented the key correctly in the left claw, it kept fumbling with it until it was correctly oriented in the right claw.

@ThatGMoney @GaryMarcus I prefer the libertarian approach of letting companies hire who they want. If their approach is truly biased, then another company would make a lot of money by hiring those great people that the other companies are ignoring.