> Does ParseHub respect robots.txt?
> We're working on an admin panel to give webmasters full transparency and control. We'll have more info soon.
What does that mean? I read it as ParseHub does not respect robots.txt, which as a content owner is a bit disappointing. Would you elaborate on your thinking?
One question about this FAQ:
What does that mean? I read it as ParseHub does not respect robots.txt, which as a content owner is a bit disappointing. Would you elaborate on your thinking?