Comment

Comment by ThePhysicist

In my understanding what Cliqz did, at least in the beginning, was to buy clickstream data and then build an index on top of that. So in a sense they just scraped Googles' search index, as almost all users rely on Google for finding stuff on the web. The clickstream data gives you both the search query and the website(s) users visit after searching, so it's pretty easy to build a search index from that, at least for popular searches (it might be more difficult for the long tail of search queries).

A lot of the clickstream data you can buy comes from browser extensions btw, and often gets collected without users knowing about it (looking at you, "Web of Trust"). I think their reliance on such data was the reason Cliqz acquired Ghostery, which also collects a copious amount of "anonymous" data from its users. On one hand it's a neat idea since you're basically standing on Googles' shoulders, on the other hand it's at least questionable for a "privacy-first" company as the generation of the search index is based on personal data mined from (often unwitting) users.

That said I don't know how their system evolved, so maybe today they have another way to build their index.

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton