Comment by Grimm1

Only 2 companies in the US have an independent English based index with the contents of the entire web. Granted, the sheer volume of data is a barrier to making the index but removing that, only 4 US companies have crawled the entire internet. I'm going to have to disagree with you on that one. To write a crawler capable of the scale and timeliness to crawl the entire web in a week or two requires some pretty solid engineering. I don't however disagree that building a good search is also difficult.