Comment Comment by vindin robots.txt is merely a gentleman’s courtesy at this point. Nobody is obligated to follow it. parent vindin 5 months ago