Comment by jsheard

Real search engines respect robots.txt so you could just tell them not to enter Markov Chain Hell.