Comment Comment by CallMeMarc If I remember correctly they changed their policy quite some time ago and started to ignore the robots.txt, but not 100% sure about it parent CallMeMarc 2 years ago