Comment by ProAm

Id argue its their job to index the internet, not dictate how people build their webpages to determine the order they are displayed. They want their crawler to be faster and to capture market share/screen time.