Comment Comment by daoudc Yes, we veto sites to prevent spam.I'm not sure you would like the results of what you suggest - if you are really going to crawl everything indiscriminately, you will end up with a lot of rubbish. Just check out Common Crawl if you want to get an idea of what it would look like. parent daoudc 2 years ago
I'm not sure you would like the results of what you suggest - if you are really going to crawl everything indiscriminately, you will end up with a lot of rubbish. Just check out Common Crawl if you want to get an idea of what it would look like.