Day one of "writing a reasonable search engine" would be to kill sitespam, no? Any time you see N sites with the same or very similar content, you should assume immediately drop the rank of all but a small slice at the top. You can choose to keep the one with most trusted incoming link (regular pagerank) but also you could just trust the oldest one in your index!
For example: there is now an epidemic of StackOverflow clone sites. They just post the SO answers with their own ads. But I don't want that site. So how on earth can Google show the clone sites on top of the true StackOverflow?
You'd think they have systems in place with hundreds of thousands of canary queries and "known expected rankings" such that IF one of the fraud sites manage to trick their system, they can just swiftly patch it to restore order and bury the clone sites after page 100. But no.
For example: there is now an epidemic of StackOverflow clone sites. They just post the SO answers with their own ads. But I don't want that site. So how on earth can Google show the clone sites on top of the true StackOverflow?
You'd think they have systems in place with hundreds of thousands of canary queries and "known expected rankings" such that IF one of the fraud sites manage to trick their system, they can just swiftly patch it to restore order and bury the clone sites after page 100. But no.