> Since Google uses multiple IP addresses to crawl it’s also difficult to block these type of GET flood
It wouldn't be too hard to block by User-Agent: Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html); if you notice the traffic.
Feedfetcher does not fetch robots.txt though; so you'd have to do something in your server config.
It wouldn't be too hard to block by User-Agent: Mozilla/5.0 (compatible) Feedfetcher-Google; (+http://www.google.com/feedfetcher.html); if you notice the traffic.
Feedfetcher does not fetch robots.txt though; so you'd have to do something in your server config.
[edit: fixed a typo, and agree with the update]