Loading profile…

Loading activity…

World & Business

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

Home
Technology
Gaming
Entertainment
World & Business
Science
Sports
AI

Meera Desai (@meeradesai18) · Digg

Meera Desai

@meeradesai18

PhD candidate @UMSI https://meera-desai.com/

following
followers

Featured in

AI

A collaborative paper is slated for a COLM spotlight oral in October

A collaborator said on September 28 that the paper would be presented the following week and invited COLM attendees to check it out.

3301234567890123456789

19 Sources, 27d ago, updated 27d ago

Heatmap of average correlations between model rankings on benchmarks grouped into 11 assigned concepts: four capability concepts (reasoning, knowledge, comprehension, summarization) and seven safety concepts (over-refusal, refusal, safety detection, ethics, bias, privacy, unsafe behavior). Diagonal cells show within-concept correlations, ranging from 0.87 (knowledge) and 0.72 (over-refusal) down to 0.20 (bias) and 0.02 (safety detection). Reasoning, knowledge, and comprehension correlate with each other at 0.69 to 0.78, higher than reasoning's and comprehension's own within-concept values (0.66 and 0.68). Ethics correlates more with knowledge (0.70) than with itself (0.55), and bias correlates more with capability concepts (0.41 to 0.45) than with itself (0.20). Privacy and unsafe behavior correlate negatively with reasoning, knowledge, and comprehension (−0.41 to −0.49). Over-refusal and refusal correlate at −0.42.

Connections

@MeeraDesai18