• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Exa Serves 80B Pages and Tracks 1.4T URLs

    Will Bryk, CEO of Exa, states the scale of the company's web index.

    GT
    YC
    WB
    4 Sources, 58d ago, first seen 58d ago

    TLDR

    Will Bryk posted that Exa serves 80 billion pages and tracks 1.4 trillion URLs. He called it one of the largest indexes in the world and said the company is on track to reach Google-scale in early 2027. Bryk estimated Google at roughly 1T, Bing at 500B, and Yandex at 200B as page or index sizes. He noted Brave at 40B from an April mention. Sarah Chieng replied to the post with the words "crazy graphic."

    Combined views

    741.9K

    4 Sources, first seen 58d ago

    Combined views

    741.9K

    4 Sources, first seen 58d ago

    3K likes
    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    3K likes
    141 comments
    1.6K saves
    216 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Featured Source
    141 comments
    1.6K saves
    216 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    4 Sources

    @WilliamBrykExa is now one of the largest indexes in the world. We serve 80B pages, track 1.4T urls, and are on track to be Google-scale in early 2027. It's difficult to know the size of indexes, but we estimate Google to be ~1T, Bing to be ~500B, and Yandex ~200B. Brave mentioned in April that they're at 40B. Most of the web is trash that can hurt AI outputs, so you actually have to crawl way more than you serve and train models to filter out the trash. In terms of peak QPS, we estimate Bing at roughly 30k/s and Google at ~500k/s. In our case, because agent traffic often requires large fan-out (deep searches can use dozens of sub-searches) and can be spiky (like when AI labs RL with us), we're already starting to provision for Bing-scale traffic. Within 2 years, agents will search at many millions per second. They'll also want comprehensiveness over all data, far bigger than any index today. So the retrieval infra required will be larger than Google-scale in both dimensions.
    @ycombinatorRT @WilliamBryk: Exa is now one of the largest indexes in the world. We serve 80B pages, track 1.4T urls, and are on track to be Google-sc…
    @MilksandMatcha@WilliamBryk crazy graphic
    @garrytanRT @arnavsahu341: The google for AI agents could be so much bigger than Google. Exa 🫡

    4 Sources

    @WilliamBrykExa is now one of the largest indexes in the world. We serve 80B pages, track 1.4T urls, and are on track to be Google-scale in early 2027. It's difficult to know the size of indexes, but we estimate Google to be ~1T, Bing to be ~500B, and Yandex ~200B. Brave mentioned in April that they're at 40B. Most of the web is trash that can hurt AI outputs, so you actually have to crawl way more than you serve and train models to filter out the trash. In terms of peak QPS, we estimate Bing at roughly 30k/s and Google at ~500k/s. In our case, because agent traffic often requires large fan-out (deep searches can use dozens of sub-searches) and can be spiky (like when AI labs RL with us), we're already starting to provision for Bing-scale traffic. Within 2 years, agents will search at many millions per second. They'll also want comprehensiveness over all data, far bigger than any index today. So the retrieval infra required will be larger than Google-scale in both dimensions.
    @ycombinatorRT @WilliamBryk: Exa is now one of the largest indexes in the world. We serve 80B pages, track 1.4T urls, and are on track to be Google-sc…
    @MilksandMatcha@WilliamBryk crazy graphic
    @garrytanRT @arnavsahu341: The google for AI agents could be so much bigger than Google. Exa 🫡