Comment by dbetteridge

I notice they don't actually give a good reason that robots.txt isn't suitable.

Change for the sake of it?