/ai

BRIDGE ASR 2.0’s test of multilingual speech recognition

A user recommends Humyn Labs’ benchmark, which they say tests 23 models on 10–15-minute, two-person conversations in 18 Indic languages plus Spanish, Portuguese and Vietnamese.

EL

1 Source, 1h ago, first seen 1h ago

TLDR

A user recommends BRIDGE ASR 2.0 for evaluating speech recognition in real conversations, where people pause, talk over each other and switch languages. They highlight code-switch F1, which checks whether English words mixed into an Indic sentence stay in English: writing “data backup” in Devanagari scores zero, they say. According to the post, the leaderboard can be filtered by language and metric, and the methodology and evaluation data are public.

Today's Rank

—

Not ranked yet

Combined views

2.9K

1 Source, first seen 1h ago

1 Source

@omarsar0Recommended benchmark. I expect voice to become one of the main ways people interact with robots and physical AI. That makes speech recognition in real conversations much more important than it seems today. Real conversations are hard to transcribe. People pause, talk over each other, and switch languages mid-sentence. Robots also need to understand the languages people actually speak. More than 5.5 billion people across the Global South are non-English speakers. The community needs a good way to measure these frontier capabilities. @humynlabs built BRIDGE ASR 2.0 to measure exactly this. It tests 23 speech recognition models on real two-person conversations, each 10 to 15 minutes long, in 18 Indic languages plus Spanish, Portuguese, and Vietnamese. The metric I find most useful is code-switch F1. It checks whether English words mixed into an Indic sentence stay in English. A model that writes "data backup" in Devanagari script scores zero. You can also filter the leaderboard by language and by metric. The methodology and evaluation data are public, and the team wants researchers to test the benchmark and find where it breaks. Check out the benchmark here: https://humynlabs.ai/bridge/ASR/2.0

Today's Rank

—

Not ranked yet

Combined views

2.9K

1 Source, first seen 1h ago

9 likes

Today's Rank

—

Not ranked yet

Today's Rank

—

Not ranked yet

9 likes

15 comments

6 saves

2 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

TLDR

A user recommends BRIDGE ASR 2.0 for evaluating speech recognition in real conversations, where people pause, talk over each other and switch languages. They highlight code-switch F1, which checks whether English words mixed into an Indic sentence stay in English: writing “data backup” in Devanagari scores zero, they say. According to the post, the leaderboard can be filtered by language and metric, and the methodology and evaluation data are public.

15 comments

6 saves

2 reposts

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

Sentiment

Positive——Negative

Summary

Not enough discussion yet.

No sentiment analysis available yet.

1 Source

@omarsar0Recommended benchmark. I expect voice to become one of the main ways people interact with robots and physical AI. That makes speech recognition in real conversations much more important than it seems today. Real conversations are hard to transcribe. People pause, talk over each other, and switch languages mid-sentence. Robots also need to understand the languages people actually speak. More than 5.5 billion people across the Global South are non-English speakers. The community needs a good way to measure these frontier capabilities. @humynlabs built BRIDGE ASR 2.0 to measure exactly this. It tests 23 speech recognition models on real two-person conversations, each 10 to 15 minutes long, in 18 Indic languages plus Spanish, Portuguese, and Vietnamese. The metric I find most useful is code-switch F1. It checks whether English words mixed into an Indic sentence stay in English. A model that writes "data backup" in Devanagari script scores zero. You can also filter the leaderboard by language and by metric. The methodology and evaluation data are public, and the team wants researchers to test the benchmark and find where it breaks. Check out the benchmark here: https://humynlabs.ai/bridge/ASR/2.0