The popular evaluation resource receives 500,000 views per month.
The core Massive Text Embedding Benchmark (MTEB) team has heavily updated their leaderboard Space.
It's so, so much faster, and there's much more information about the best embedding models (and related) to glean from it.
Details in 🧵
Many users are excited about the MTEB leaderboard upgrade on Hugging Face because the faster performance, visual blogpost, and practical improvements make it far more useful and enjoyable.
The MTEB Leaderboard is viewed ~500K times/month & just got a major upgrade: https://hf.co/spaces/mteb/leaderboard
Great work by @risolomatin Kenneth Enevoldsen @isaacchung1217 & team!
@din0s_ @tomaarsen I can finally ditch my pandas scripts
@tomaarsen sub second mteb space load, AGI is officially here
Check out the blogpost here: https://huggingface.co/blog/Samoed/mteb-v3-leaderboard
It's a very visual blogpost, it's a joy to read (and/or look) through.
You can now more easily distinguish model types (dense/static/late-interaction embedding, reranker), pin models for easier comparison, get per-dataset explanations + dataset previews, head-to-head comparisons, and more.
Download the CSV results, or even access it via API.
The main improvement is the speed. Previously, loading a new benchmark leaderboard or applying a filter would be extremely slow. Now, it's virtually realtime to browse and filter.
I'm very excited about this update, great job to Roman Solomatin who led this work, as well as Kenneth Enevoldsen and Isaac Chung.
Now, go check out the updated leaderboard here: https://huggingface.co/spaces/mteb/leaderboard
@tomaarsen Ah yes that’s much faster!
@tomaarsen Wow, this is so much faster. Really good work!
@din0s_ I know right, it feels sooo good
@Manuel_NLP Roman killed it!
@E_Rijgersberg Night and day indeed, I'm very happy with it
@Muennighoff @risolomatin @isaacchung1217 500k views is wild for a leaderboard. nice upgrade, needed it honestly
