Comment by jenadine

Not a lawyer, but my understanding is that crawling is a gray area. By putting that into their terms of service that most of the planet is agreeing to, they protect themselves in the case someone complains.

Also, this is about crawling and has little to do with the license of the contents.