Joined 14 years ago
Why should I be treated differently than search engine spiders?
If somebody doesn’t want their site scraped then they can let people know with robots.txt. Get off your high horse.
Do you by chance consider yourself a conservative?
Why should I be treated differently than search engine spiders?
If somebody doesn’t want their site scraped then they can let people know with robots.txt. Get off your high horse.