_pjee

Thanks for the interest and feedback everybody. I won't be seriously having conversations about this or working on it until summer.

Given I'm going to take a completely different approach that doesn't use anything proprietary, and non-competes aren't enforceable in California, I'd assume this is ok?

Yeah I was thinking the same thing.

I was looking into that, looks like they have an API but it's limited to 10k requests per day, and TOS says you can't blend the results with other sources.

Nice! I was just hoping to beat regex scrapers.

Pefect. I joined, thank you.

I'd imagine the search engine of the future will be like that. Let a summarization AI do the search and then report back on results, with citations.

I think one issue is broadness. When a user issues a query there is often a choice of 1) low quality material that directly answers the specific question, or 2) high quality material about the subject more broadly. If someone asks "how do I fix water in camera iphone 3", if someone has set up a content farm for that exact question Google will probably send the user there, rather than to a broader repair page from apple.com. There's tradeoffs in search ranking, which is why I think it's possible to make a search engine that takes the opposite side of the tradeoffs that Google has.

Nice job on marginalia.nu btw. Your site is a good reminder that there is a whole internet out there that just isn't able to rank on Google for various reasons.

If it's useful it isn't a gimmick.

It depends on your goal. If the focus of your search engine is question answering (like Google), ChatGPT is serious competition. Browsing the web to discover online communities and niche websites is an entirely different activity that ChatGPT doesn't compete with, and frankly Google doesn't do well either.

Been hearing a lot about Kagi on here, might reach out. My concern is that they're trying to beat Google at its own game by being better at scraping, document understanding and question-answering.

Hacker news doesn't have dm right?