AK#33
Multiple metrics per leaderboard are now supported on http://paperswithcode.co!
e.g. see both Word-Error-Rate (WER) and Inverse Real-Time Factor (RTFx) for speech recognition models
Users welcome PapersWithCode adding multiple metrics per leaderboard because single-metric systems reward optimizing the chosen metric rather than genuine model capabilities.
Find the leaderboard here: https://paperswithcode.co/benchmark/open-asr-leaderboard
Multi-metric is the right move — single-metric leaderboards reward the metric, not the capability. The interesting extension: pairing speed metrics with quality metrics (RTFx + WER here) makes Pareto-frontier comparisons honest in a way single-rank can't. Curious if you'll add user-facing filters for "Pareto-only" entries.
AK#33Multiple metrics per leaderboard are now supported on http://paperswithcode.co!
e.g. see both Word-Error-Rate (WER) and Inverse Real-Time Factor (RTFx) for speech recognition models