Loading profile…

Loading activity…

World & Business

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

Hanna Wallach (@hannawallach.bsky.social) (@hannawallach) · Digg

Hanna Wallach (@hannawallach.bsky.social)

@hannawallach

Microsoft Research NYC. Fairness, accountability & transparency in AI/ML. NeurIPS & ICML board member, WiML co-founder, sloth enthusiast. She/her.

following
followers

Featured in

AI

Do AI benchmarks measure what they claim to?

Stanford HAI says it supported two studies examining tests whose scores influence which AI models get funded, bought and regulated.

5001234567890123456789

7 Sources, 15d ago, updated 15d ago

AI

A collaborative paper is slated for a COLM spotlight oral in October

A collaborator said on September 28 that the paper would be presented the following week and invited COLM attendees to check it out.

3301234567890123456789

19 Sources, 27d ago, updated 27d ago

Heatmap of average correlations between model rankings on benchmarks grouped into 11 assigned concepts: four capability concepts (reasoning, knowledge, comprehension, summarization) and seven safety concepts (over-refusal, refusal, safety detection, ethics, bias, privacy, unsafe behavior). Diagonal cells show within-concept correlations, ranging from 0.87 (knowledge) and 0.72 (over-refusal) down to 0.20 (bias) and 0.02 (safety detection). Reasoning, knowledge, and comprehension correlate with each other at 0.69 to 0.78, higher than reasoning's and comprehension's own within-concept values (0.66 and 0.68). Ethics correlates more with knowledge (0.70) than with itself (0.55), and bias correlates more with capability concepts (0.41 to 0.45) than with itself (0.20). Privacy and unsafe behavior correlate negatively with reasoning, knowledge, and comprehension (−0.41 to −0.49). Over-refusal and refusal correlate at −0.42.

Connections

@hannawallach