AI Judge changed title after evaluation, original title: "Perplexity AI releases pplx-embed-v1-late-0.6b, a 0.6B-parameter token-level embedding model optimized for late-interaction retrieval with MaxSim scoring"
Companion kernel delivers 3-5x speedup on Metal and CUDA.
Julien Chaumond#335
Releasing my first kernel on @huggingface: MaxSim
Late-interaction retrieval (ColBERT / PyLate) bottlenecks on materializing the full similarity matrix. This kernel avoids it by using tiled scoring with simdgroup_matrix (Metal) and WMMA.
Result is 3–5× speedup compared to naive PyTorch.
Try it out 👇
Users are excited about Perplexity open-sourcing its multilingual ColBERT embedding model with the optimized MaxSim Kernel on Hugging Face because the 3-5x speedup and multilingual support fill a practical gap for retrieval tasks.
okay maybe it's a good time? We have a small colbert model trained at pplx, it is a continue-training of pplx-embed-0.6b, so native multilingual, just made it open and added a section how to use MaxSim kernel:
https://huggingface.co/perplexity-ai/pplx-embed-v1-late-0.6b
Releasing my first kernel on @huggingface: MaxSim
Late-interaction retrieval (ColBERT / PyLate) bottlenecks on materializing the full similarity matrix. This kernel avoids it by using tiled scoring with simdgroup_matrix (Metal) and WMMA.
Result is 3–5× speedup compared to naive PyTorch.
Try it out 👇
We causally trained a lot of SOTA search models internally, shall we make some small release from time to time 🤣🤣
@bo_wangbo stealth releasing probably the strongest open multilingual ColBERT (and it's an encoder-based one 🫶)
Very happy to see this, I've played with @perplexity_ai's Qwen based encoder in PyLate, it's really cool to see it works just with `trust_remote_code=True`!
oh! cool to see @perplexity_ai train late interaction (colbert) models
okay maybe it's a good time? We have a small colbert model trained at pplx, it is a continue-training of pplx-embed-0.6b, so native multilingual, just made it open and added a section how to use MaxSim kernel:
https://huggingface.co/perplexity-ai/pplx-embed-v1-late-0.6b
@bo_wangbo stealth releasing probably the strongest open multilingual ColBERT (and it's an encoder-based one 🫶)
Very happy to see this, I've played with @perplexity_ai's Qwen based encoder in PyLate, it's really cool to see it works just with `trust_remote_code=True`!
okay maybe it's a good time? We have a small colbert model trained at pplx, it is a continue-training of pplx-embed-0.6b, so native multilingual, just made it open and added a section how to use MaxSim kernel:
https://huggingface.co/perplexity-ai/pplx-embed-v1-late-0.6b
Multilingual colbert model from @bo_wangbo at perplexity, trained with PyLate, we are finally getting strong open-source multilingual colbert models
okay maybe it's a good time? We have a small colbert model trained at pplx, it is a continue-training of pplx-embed-0.6b, so native multilingual, just made it open and added a section how to use MaxSim kernel:
https://huggingface.co/perplexity-ai/pplx-embed-v1-late-0.6b
@ErikKaum @huggingface nice! i worked on something similar last year(including the backward pass in triton) never got to publish it. time to revisit ig!
@ErikKaum @huggingface TIL about kernels. Cool initiative
@antoine_chaffin @bo_wangbo @perplexity_ai I found out about it this weekend, so reimplemented it (along with your colbert zero and late on models, and neobert and openai privacy filter). After oai privacy filter, the bidirectional attention mask on qwen3 gave me another idea…
@ErikKaum @huggingface Congrats, way to go!!
@antoine_chaffin @perplexity_ai my personal faith: every language model should be multilingual by default 🤣
@ErikKaum @huggingface Very cool
@bo_wangbo awesome to see this 🔥
also: I really need to make the backward implementation of this as well 👀
@staghado @huggingface Nice, I highly recommend getting back to it, kernel dev tooling is a lot nicer than it was a year ago 😄
I'm also planning a backward pass for this one, for maxsim backwards is probably even more important that forward 👍
@huggingface cc at least @antoine_chaffin & @lateinteraction
@ErikKaum @huggingface beautiful!
@ErikKaum @huggingface will definitely give it a try! iirc i hit an issue matching pytorch's 'first index wins' tiebreak in triton, because i couldn't control the order in which the argmax reductions were executed in. It's mathematically correct to take any argmax index but i wanted to match pytorch.
@ErikKaum @huggingface How do you create kernel on hf ?
@ErikKaum @huggingface oa that sounds sick! Congrats on the release!
@PITTI_DATA @bo_wangbo @perplexity_ai Nice!! Did you ever try to use a LI model as base for token classification?
@ErikKaum @huggingface I've one tagged as model that I should maybe move to a kernel https://huggingface.co/blanchon/voronoi
