Comment Comment by rsbadger Exactly - seems the safest way is to explicitly block known bots by user agent from even reaching pages you don't want indexed. parent rsbadger 4 years ago