/Tech32d ago

Waterloo researchers replicate DPR model in Pyserini

Researchers Xueguang Ma, Rodrigo Pradeep and University of Waterloo undergraduate Kai Sun replicated the 2020 Dense Passage Retrieval model from Karpukhin et al. Their April 2021 arXiv study integrated BM25 sparse retrieval with dense neural retrievers inside the Pyserini toolkit. The resulting hybrid methods appear in Pyserini and Pi-Serini, with evaluations on BEIR, LongEmbed and BrowseComp-Plus benchmarks. Recent extensions pair the retriever with search agents.

142633512132.1K

#301

Original post

Jimmy Lin@lintool#859inTech

I think @xueguang_ma is being too modest, so I'll provide context: he along with @rpradeep42 and a UWaterloo ugrad (Kai Sun) popularized hybrid search in its current form.

So, if you're using hybrid search today, thank them. 🙏

Yes, this is clickbait-y, so I'll support my claims 🧵

Xueguang Ma@xueguang_ma

This plot reminds me of my first IR work reproducing DPR in Pyserini, where we found BM25 is amazingly helpful when hybrid with a dense retriever. BM25 is never just a simple baseline -- used the right way, it can easily outperform many fancy methods. BM25 was the most robust method shown in BEIR, the most effective and efficient method for long-context search shown in LongEmbed, and now @mattjustram and @xuzihuan4 show that BM25 can push the search agents into the best efficiency frontier.

p.s. Pyserini and pi-serini are two different repos.

10:08 AM · May 14, 2026 · 4.9K Views

Sentiment

Many users praise hybrid BM25 search methods for modern retrieval systems and agentic loops because they deliver strong efficiency gains while crediting UWaterloo innovations like reciprocal rank fusion as a practical breakthrough.

Pos

100.0%

Neg

0.0%

4 comments with sentiment.

Cluster Engagement

Posts from X

Most Activity

VIEWS5.9KBOOKMARKS49LIKES86RETWEETS12REPLIES3

Jimmy Lin@lintool

Since we're counting model parameters, let me introduce you to a two-parameter model for agentic search that's awesome: It's called BM25. I haven't tried it yet, but I think fp4 will work fine. https://arxiv.org/abs/2605.10848

31d5.9K8649

Leo Boytsov@srchvrs

BTW, there's truly unique (to my knowledge) @UWaterloo invention that has become a largely essential cog in of the hybrid retrieval machine. Yet nobody is talking about it. It is a reciprocal rank fusion published by G. Cormack, @claclarke , and Stefan Büttcher. AFAIK, it is implemented and used nearly everywhere.

Jimmy Lin@lintool

Thus, our conclusions: This I believe is the first demonstration of the need for hybrid search. Hence the claim that hybrid search is a @UWaterloo innovation. You're welcome!

The broader lesson is that old baselines are still surprisingly important. Let's not forget them.

32d2.1K229

Leo Boytsov@srchvrs

The DPR paper might have shown you wouldn't need BM25, but that neural models were insufficient was known before the popularization of hybrid search. In particular, in our SIGIR paper (that slightly preceded the BEIR paper by @beirmug and colleagues) we learned that strong neural rankers don't generalize well and underpform BM25. @beirmag paper makes some additional findings, in particular, that the dense retrieval generalizes more poorly compared to the BERT-based rankers. The @UWaterloo team surely deserves the credit for the ultimate demonstration of the usefulness of the hybrid search. However, the need for a hybrid search was also motivated by prior work. https://arxiv.org/abs/2103.03335

Jimmy Lin@lintool

Thus, our conclusions: This I believe is the first demonstration of the need for hybrid search. Hence the claim that hybrid search is a @UWaterloo innovation. You're welcome!

The broader lesson is that old baselines are still surprisingly important. Let's not forget them.

32d1.5K184

Andrew Drozdov@mrdrozdov

@xueguang_ma As a counter point, I want to share a figure from @mat_jacob1002 's research when interning with us. He found that bm25 + pointwise reranker is sometimes worse than a dense embedding alone. https://arxiv.org/abs/2411.11767

32d46171

Xueguang Ma@xueguang_ma

p.s. Pyserini and pi-serini are two different repos.

Jheng-Hong Yang@mattjustram

someone already wrote a love letter to pi, by @badlogicgames.

so we wrote a love paper to pi :)

with my teammates @xuzihuan4 and @lintool.

a few days ago, i promised i’d share some fun plots once Pi-Serini joined the BrowseComp-Plus deep research agent party.

now, it’s about time.

here weeeee goooooo.

bear with the sloppy images first. the serious one is at the end. the question was simple:

how far can we push deep research with BM25 + pi?

turns out: weirdly far.

33d11.4K6643

Jheng-Hong Yang@mattjustram

@xueguang_ma just a warning that this is the sloppy image i made, you can find serious plot & numbers in our paper :) https://arxiv.org/abs/2605.10848

33d23132

Jimmy Lin@lintool

But that's not what we found: even with DPR, a dense-sparse hybrid with BM25 is significantly better than DPR alone. https://arxiv.org/abs/2104.05740

Jimmy Lin@lintool

The original DPR paper https://aclanthology.org/2020.emnlp-main.550/ claimed that with dense retrieval, you no longer needed BM25.

32d65551

Jimmy Lin@lintool

But I think we can do better... what about zero parameters? Let me introduce you to something else that's awesome: It's called grep. https://arxiv.org/abs/2605.05242

Jimmy Lin@lintool

31d39661

Jimmy Lin@lintool

Thus, our conclusions: This I believe is the first demonstration of the need for hybrid search. Hence the claim that hybrid search is a @UWaterloo innovation. You're welcome!

The broader lesson is that old baselines are still surprisingly important. Let's not forget them.

Jimmy Lin@lintool

But that's not what we found: even with DPR, a dense-sparse hybrid with BM25 is significantly better than DPR alone. https://arxiv.org/abs/2104.05740

32d3.9K90

Jimmy Lin@lintool

The original DPR paper https://aclanthology.org/2020.emnlp-main.550/ claimed that with dense retrieval, you no longer needed BM25.

Jimmy Lin@lintool

I think @xueguang_ma is being too modest, so I'll provide context: he along with @rpradeep42 and a UWaterloo ugrad (Kai Sun) popularized hybrid search in its current form.

So, if you're using hybrid search today, thank them. 🙏

Yes, this is clickbait-y, so I'll support my claims 🧵

32d55121

Luca Soldaini 🎀@soldni

@lintool cheeky not to count index as its embeddings

Jimmy Lin@lintool

30d45450

Ronak Pradeep@rpradeep42

@srchvrs @UWaterloo @claclarke Every TREC deck I’ve made has eventually converged to this slide:

Leo Boytsov@srchvrs

32d25920

Andrew Drozdov@mrdrozdov

@xueguang_ma @mat_jacob1002 I remain a bm25 stan! I guess I also subscribe to "there's no silver bullet". In many real world datasets the lexical overlap between a 512-token chunk and a query can be very small, but how you chunk and augment your data can enable you to bridge this gap.

32d111

Leo Boytsov@srchvrs

@mrdrozdov @xueguang_ma @mat_jacob1002 I think in that case one should blame the ranker, not a hybrid search. With a better ranker, hybrid search + ranker should have outperformed just vector search.

Andrew Drozdov@mrdrozdov

32d2700

Charlie Clarke@claclarke

@srchvrs @UWaterloo 🙏

32d793

Andrew Drozdov@mrdrozdov

@srchvrs @xueguang_ma @mat_jacob1002 Indeed. No silver bullet. :)

Leo Boytsov@srchvrs

@mrdrozdov @xueguang_ma @mat_jacob1002 I think in that case one should blame the ranker, not a hybrid search. With a better ranker, hybrid search + ranker should have outperformed just vector search.

32d2510