Fastly warns AI bots can hit sites 39K times per minute

This article got me thinking -- I know personally on some of my personal sites and other sites I run, I've been seeing a metric ton of traffic from AI bots. I even block some of the bot traffic using the "standard" methods. While not as common, I've seen quite a few sites that people run on Shared Web Hosting suspended as of late, and while only speculation, it seems like being constantly hit by AI bot traffic, they're going through their monthly limit of bandwidth and their accounts are getting suspended. It will be interesting how Cloudflare, Fastly, and others iterate on the tech to help block bots (https://developers.cloudflare.com/bots/additional-configurations/block-ai-bots/) and at this point RFC 9309 (https://datatracker.ietf.org/doc/html/rfc9309) hasn't been updated. But even implementing a robots.txt file, you're still relying on the "AI bot" operators to respect this implementation, if you have done so on your site.