Comment Comment by ceejayoz Sitemaps (http://www.sitemaps.org/) help a bit in that regard, as crawlers can check the sitemap and only crawl updated content. parent ceejayoz 13 years ago
