Comment by GhostVII

It also makes sense because they can't crawl a page that is behind a paywall.