chris_f
Joined 6 years ago
Karma
1,208
Submissions
249
Member since
Feb 19
Was working on https://www.runnaroo.com
View on Hacker NewsRecent submissions
Comment · 4 years ago
Any specific sites? Happy to spin one of these up for you focused on Web3. These work the best when the search engine creator has domain expertise to showcase the best sources.
I could try to find some sources, but my guess at the best Web3 sites would probably miss the mark.
Comment · 4 years ago
That's great! And I think the results even improve a little more if you just search 'distributed model parallel' without 'pytorch' [0]
So still some work to do, but I've personally found this type of search engine really valuable for some of my interests where I am able to curate the sites that might not normally rank, but I know are higher quality.
[0] https://you.com/niche/pytorch?q=distributed+model+parallel
Comment · 4 years ago
Yeah, that's a good point. As new ones get built, we will need a way to make them discoverable.
Also, just as a heads up, there is a link at the bottom of the search page [0] if anyone would like to build one of these search pages on any topic that might be interesting. It's just a form now bc we are working through the process.
[0] https://form.asana.com/?k=13NZtQkfNVw3CTE1jxngvg&d=119356944...
Comment · 4 years ago
Good stuff! Github is one of the sources, but not specific repos. I actually think we can break out Github into individual repo sources pretty easily.
Comment · 4 years ago
In the broadest sense, a highly targeted search engine (like this) can provide better results because Google has to determine user search intent AND return the right results from trillions of webpages. The advantage of this search engine is that all of the users are looking for the same type of information, and the result sources can be curated to ensure high quality and relevant results.
The more targeted the topic, the harder the time Google has to provide quality signal through the noise of SEO and sheer volume of content on the web.
In addition to the above, the UI provides some cool features like allowing horizontal scrolling of sources to provider higher information density (important for discovery), and some source content can be viewed in the side pane without leaving the page.
But ultimately it would be good to hear if this approach does make it easier to find relevant and higher quality Pytorch info.
Comment · 4 years ago
Appreciate the feedback, it is helpful! The relevance tuning is a continuous work in progress.
Comment · 4 years ago
Comment · 4 years ago
Worth mentioning is the Alexandria.org project [0]. It is a non-profit search engine built on data from Common Crawl. The coverage is limited because of Common Crawl, but the relevance is decent. They also provide an API.
I believe one of the biggest impacts toward breaking up Google's monopoly on search is making them open up access to their index, even requiring Google to provide direct API search access for others to build alternative search products. They have a search API today, but it is prohibitively expensive to build on ($5/1000 calls).
I built a fairly popular search engine a couple years back, but the cost of Google's search API and increasing number of bot attacks make it difficult to reason keeping it online.
Comment · 4 years ago
I just finished reading the PaLM [0] blog post and being genuinely impressed with all the NLP advancements. This search result was a fun juxtaposition.
SERP: https://imgur.com/a/YbEtrLZ
Note: The answer 9.6076 is not wrong. It's for an imperial gallon, but it is an odd results being in the US.
[0] https://ai.googleblog.com/2022/04/pathways-language-model-pa...
Comment · 4 years ago
Mojeek is excellent, and because they use 100% their own index they have a much higher hill to climb.
When I say "Best"for a general search engine, my definition is that it would fulfill the needs of myself and my non-technical family members. Kagi and Brave Search both do that while being different enough to not be just another Bing clone. I use Mojeek often, think it is great, and having their own index is a tremendous asset, but it doesn't quite meet that full definition yet.
Comment · 4 years ago
Credit to them for trying some new things on the UI front, but it looks like the organic results are from Bing (like most other alt/privacy search engines). It would be interesting to learn more about how/if they plan to build their own index, or set themselves apart.
IMO, Kagi and Brave search are the two best alternative general search engines right now.
Runnaroo was pretty good as well ;-)