Comment by seanlinmt

While that's great, there should be instances where crawlers should ignore noindex directives. For example, all .gov sites.