Comment by berendhh

One Question: What robots.txt do you have to obey? http://app.imdb.com/robots.txt or http://www.imdb.com/robots.txt?

The second one is rather restrictive (and the first one does not exist...)