• Home
  • Technology
  • Gaming
  • Entertainment
  • World & Business
  • Science
  • Sports
  • AI
HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
  • HomeTechnologyGamingEntertainmentWorld & BusinessScienceSportsAI
    • Home
    • Technology
    • Gaming
    • Entertainment
    • World & Business
    • Science
    • Sports
    • AI
    AI

    Pieter Levels Struggles Building Google SERP Scraper

    Indie founder details repeated failures scraping Google results for Hotelist despite proxies and solvers.

    @L
    1 Source, 26d ago, first seen 26d ago

    TLDR

    Pieter Levels posted about weeks of attempts to create his own scraping tool comparable to ScrapingBee. He explained that AI has repeatedly indicated it cannot complete the required functionality. The main requirement involves pulling search engine results pages from Google for his Hotelist project. Even after adding residential IP proxies, CAPTCHA solving services, and OpenSERP, the tool fails to handle thousands of queries without getting blocked. Levels described the persistent issue as part of his solo development process for data collection needs.

    Combined views

    160.9K

    1 Source, first seen 26d ago

    Combined views

    160.9K

    1 Source, first seen 26d ago

    1.1K likes
    1.1K likes
    100 comments
    1.5K saves
    21 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    100 comments
    1.5K saves
    21 reposts

    Sentiment

    Positive——Negative

    Summary

    Not enough discussion yet.

    No sentiment analysis available yet.

    Today's Rank

    —

    Not ranked yet

    Today's Rank

    —

    Not ranked yet

    1 Source

    @levelsioI've tried for weeks now to build my own @ScrapingBee and repeatedly AI has now told me that it can't do it 80% of my scraping is Google SERP results for Hotelist and even with residential IP proxies and solve CAPTCHA services and OpenSERP scraper it won't do it well for thousands of queries, it just gets blocked So I will keep paying them $99/mo A good test to see what services might remain now AGI is here My only remaining SaaS now are: - Cloudflare Domains - Cloudflare Email Sending - Cloudflare R2 Storage (for Photo AI mostly and Litestream DB backups) - Backblaze B2 for backups - Hetzner for all my servers for all my sites (VPS) - xAI for all LLMs for all my sites (so many things like moderation, writing emails, anything) - Scrapingbee

    1 Source

    @levelsioI've tried for weeks now to build my own @ScrapingBee and repeatedly AI has now told me that it can't do it 80% of my scraping is Google SERP results for Hotelist and even with residential IP proxies and solve CAPTCHA services and OpenSERP scraper it won't do it well for thousands of queries, it just gets blocked So I will keep paying them $99/mo A good test to see what services might remain now AGI is here My only remaining SaaS now are: - Cloudflare Domains - Cloudflare Email Sending - Cloudflare R2 Storage (for Photo AI mostly and Litestream DB backups) - Backblaze B2 for backups - Hetzner for all my servers for all my sites (VPS) - xAI for all LLMs for all my sites (so many things like moderation, writing emails, anything) - Scrapingbee