Comment Comment by reefoctopus Teach your students to ensure there’s a delay between requests so they aren’t hammering anyone’s server, and follow the rules in the robots.txt. I’ve scraped more than a billion pages without any issues. parent reefoctopus 7 years ago