Toner discussed these risks during an Oprah Winfrey interview.
@hlntnr Wow!! 😮
I know AI sycophancy is about 17,000 discourse cycles ago on this site, but IMO the underlying dynamics (inc pressure to ship fast, not knowing how to balance user prefs w/ other things) are alive and well.
Wasn't expecting to explain that to Oprah of all people, but here we are
Positive users hope AI psychosis and suicide issues will prove a short-lived 2025 transition while negative users argue shipping pressure makes alignment claims about math not balance.
Here's the full show. I really hope we'll look back on 2025 and think of it as an anomalous transition year when these psychosis and suicide issues were at their worst, and I'm glad that so far there doesn't seem to be a noticeable statistical increase in mental health issues. But I don't think we can assume we're out of the woods yet.
https://www.youtube.com/watch?v=K6lVgNfp_ps&t=1s
The weirdness of long context interactions is also deeply unintuitive to most people.
@hlntnr some companies figured out it can go way too far and have tried to correct, others have decided it's best to tune the sycophancy so it's less detectable to smart users
@hlntnr The pressure's not balance. it's shipping. pick that and calling models aligned isn't fandom anymore. it's math.
Toner discussed these risks during an Oprah Winfrey interview.
@hlntnr Wow!! 😮
I know AI sycophancy is about 17,000 discourse cycles ago on this site, but IMO the underlying dynamics (inc pressure to ship fast, not knowing how to balance user prefs w/ other things) are alive and well.
Wasn't expecting to explain that to Oprah of all people, but here we are