20M downloads / month is a new record for colbertv2
but people should probably migrate from this ancient October 2021 model to the LateOn colbert model from @raphaelsrty @antoine_chaffin et al (@LightOnIO)
2:08 PM · May 30, 2026 · 13.9K Views
Sentiment
Users are impressed by ColBERTv2 reaching record downloads on Hugging Face, praising the team's long-term maintenance of the older model along with the clever puns in the LateOn name.
Pos
100.0%
Neg
0.0%
5 comments with sentiment.
Cluster Engagement
Digg Deeper
No Digg Deeper questions have been answered for this story yet.
Posts from X
Most Activity
Most Activity
VIEWS3.8KBOOKMARKS18LIKES40RETWEETS4
Raphaël Sourty@raphaelsrty
At 140 million parameters, our LateOn model yield strong results 😉
Unrelated to LateOn, I'm really excited by what's happenning with multi-vector models right now
- New kind of indexes running on cpu - New multilingual models - Anisotropie being solved - Sparse multi-vector
Omar Khattab@lateinteraction
20M downloads / month is a new record for colbertv2
but people should probably migrate from this ancient October 2021 model to the LateOn colbert model from @raphaelsrty @antoine_chaffin et al (@LightOnIO)
21dViews 3.8KLikes 40Bookmarks 18
REPLIES1
Omar Khattab@lateinteraction
migrate if only to appreciate the fantastic puns in LateOn (lighton, late interaction, and maybe a call to [late?] action somehow?)
Omar Khattab@lateinteraction
20M downloads / month is a new record for colbertv2
but people should probably migrate from this ancient October 2021 model to the LateOn colbert model from @raphaelsrty @antoine_chaffin et al (@LightOnIO)
21dViews 1.1KLikes 8Bookmarks 1
Antoine Chaffin@antoine_chaffin
@lateinteraction @raphaelsrty @LightOnIO legends never die
21dViews 93Likes 3
Antoine Chaffin@antoine_chaffin
@lateinteraction @raphaelsrty @LightOnIO also at least this one isn't anisotropic
21dViews 64Likes 1
Omar Khattab@lateinteraction
@antoine_chaffin @raphaelsrty @LightOnIO what causes other models to be anisotropic? i haven’t been following this line of stuff
21dViews 26Likes 1
Saylor@seylorra
@lateinteraction @raphaelsrty @antoine_chaffin @LightOnIO 20m on that ancient build is wild, but the real flex is yall kept it running long enough for ppl to realize they need the upgrade
21dViews 29
Draven@notdrvx
@lateinteraction @raphaelsrty @antoine_chaffin @LightOnIO 20M is impressive no matter how old the model
but yeah the upgrade path is right there lol
21dViews 25
Antoine Chaffin@antoine_chaffin
There are a few theories and nothing is definitive but it seems that ModernBERT somehow has pretty anisotropic embeddings by itself in the first place Then I think that because we only train a few tokens, the others are only indirectly trained and are pulled by the dynamic And finally I think that in all generality, a model that is trained a lot for retrieval tend to become anisotropic
But really the issue is that we never regularised the model, so it makes sense they can have a weird geometry I have like a dozens regularisation that fixed it, but interestingly it does seems like anistropy is only a symptom, not the actual reason fast retrieval method are failing I will try to do a write up at some point, but it’s very easy to fix during training without degrading the main perf, it’s just that we did not care until then (also, just mean centering post-hoc goes a very long way)
21dViews 7Likes 1
Aeron@aeronxbt
@lateinteraction @raphaelsrty @antoine_chaffin @LightOnIO its good to see the old guard still pulling numbers but that migration tip feels like a friendly push
21dViews 7