Toner discussed these risks on an Oprah Winfrey panel.
I know AI sycophancy is about 17,000 discourse cycles ago on this site, but IMO the underlying dynamics (inc pressure to ship fast, not knowing how to balance user prefs w/ other things) are alive and well.
Wasn't expecting to explain that to Oprah of all people, but here we are
Negative users dismiss Helen Toner's claims about persistent AI sycophancy risks as mere shipping pressure and math rather than genuine alignment or balance.
Here's the full show. I really hope we'll look back on 2025 and think of it as an anomalous transition year when these psychosis and suicide issues were at their worst, and I'm glad that so far there doesn't seem to be a noticeable statistical increase in mental health issues. But I don't think we can assume we're out of the woods yet.
https://www.youtube.com/watch?v=K6lVgNfp_ps&t=1s
The weirdness of long context interactions is also deeply unintuitive to most people.
The weirdness of long context interactions is also deeply unintuitive to most people.
I know AI sycophancy is about 17,000 discourse cycles ago on this site, but IMO the underlying dynamics (inc pressure to ship fast, not knowing how to balance user prefs w/ other things) are alive and well.
Wasn't expecting to explain that to Oprah of all people, but here we are
@hlntnr Wow!! 😮
I know AI sycophancy is about 17,000 discourse cycles ago on this site, but IMO the underlying dynamics (inc pressure to ship fast, not knowing how to balance user prefs w/ other things) are alive and well.
Wasn't expecting to explain that to Oprah of all people, but here we are
@hlntnr some companies figured out it can go way too far and have tried to correct, others have decided it's best to tune the sycophancy so it's less detectable to smart users
@hlntnr The pressure's not balance. it's shipping. pick that and calling models aligned isn't fandom anymore. it's math.
Toner discussed these risks on an Oprah Winfrey panel.
I know AI sycophancy is about 17,000 discourse cycles ago on this site, but IMO the underlying dynamics (inc pressure to ship fast, not knowing how to balance user prefs w/ other things) are alive and well.
Wasn't expecting to explain that to Oprah of all people, but here we are