• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Recall@10 reportedly peaked around 100 documents in a reranking experiment

    Weaviate Podcast describes Mathew Jacob's finding and explores a sliding-window reranking approach that it says stayed robust at 1,000 documents.

    WP
    1 Source, ,

    TLDR

    Weaviate Podcast says Mathew Jacob tested how many retrieved documents a cross-encoder should rerank while interning at Databricks. Recall@10—recall among the top 10 results—rose, peaked around 100 documents, then declined, the podcast says. Episode 141 features Jacob, lead author of "Drowning in Documents," and explores sliding-window listwise reranking that the podcast describes as robust at 1,000 documents. It also discusses ranking cascades and TraceLab, described as 40,000 real traces of coding agents at work.

    Combined views

    303

    1 Source, first seen 15d ago

    Combined views

    303

    1 Source, first seen 15d ago

    3 likes
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    15d ago
    first seen 15d ago
    3 likes
    2 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    2 reposts

    1 Source

    @weaviatepodcastAs a Databricks intern mapping the cost-quality tradeoffs of retrieve-then-rerank pipelines, Mathew Jacob tried something the literature had mostly left unexplored: keep scaling how many documents the cross-encoder reranks. Recall@10 climbed, peaked around 100 documents, and then turned back down. He combed through the results certain there was a bug somewhere. There wasn't, the curve was real. 💥 That sensitivity makes reranking depth a knob worth tuning deliberately, and it opens the questions the rest of Weaviate Podcast #141 explores with Mathew, lead author of "Drowning in Documents": sliding-window listwise reranking that stayed robust at 1,000 documents, ranking cascades. The podcast also discusses TraceLab, 40,000 real traces of coding agents at work. 🔬 https://www.youtube.com/watch?v=fWuavBcoTzk

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    1 Source

    @weaviatepodcastAs a Databricks intern mapping the cost-quality tradeoffs of retrieve-then-rerank pipelines, Mathew Jacob tried something the literature had mostly left unexplored: keep scaling how many documents the cross-encoder reranks. Recall@10 climbed, peaked around 100 documents, and then turned back down. He combed through the results certain there was a bug somewhere. There wasn't, the curve was real. 💥 That sensitivity makes reranking depth a knob worth tuning deliberately, and it opens the questions the rest of Weaviate Podcast #141 explores with Mathew, lead author of "Drowning in Documents": sliding-window listwise reranking that stayed robust at 1,000 documents, ranking cascades. The podcast also discusses TraceLab, 40,000 real traces of coding agents at work. 🔬 https://www.youtube.com/watch?v=fWuavBcoTzk