researchers
Joined 4 years ago
Karma
63
Submissions
18
Member since
Mar 23
Recent submissions
Comment · 1 year ago
Tuning for qualitative outcomes is pretty much solved via RLHF/DPO (what this post calls "preference tuning"). Right?
Comment · 4 years ago
Thanks for reporting this. I think it has to do with certain tweets being made private/inaccessible after being added to the index. I'll look into it.
Comment · 4 years ago
A lot of that content has moved to newsletters and Twitter threads. If you know the right people to follow, there are a lot of niche, quirky, and insightful pieces that remind me of the homepages of yore.
One problem is that these things get lost in the timeline. So I made a StumbleUpon for Twitter threads [0]. Check it out!
Comment · 4 years ago
Quite tractable actually. Take a look at the Andi Search post that is on the front page right now. They say they are using several natural language models for snippet selection, question answering, and probably a few other things. These tasks are exactly the ones that we had done manually in this exercise to generate the reports.
Comment · 4 years ago
Hi HN! So to clarify to everyone, the idea is that what a concierge search engine would do, a machine could do as well.
sk55 shared this very interesting link [0] which outlines how a complex question could be broken down into micro-tasks and assembled into an answer. The interesting thing here is that each of those steps is now within the realm of automation by Transformer-based models. With the addition of a UI that encourages exploration/refinement of the query, it could be a good way to quickly gain an understanding about a broad topic.
This is one of the many theses we are exploring. Another one is that "A Google Replacement Will Not Look Like Google" [1]
[0] https://joe.cat/CHI-ka/ [1] https://re-search.xyz/writing/mapping-the-new-world-towards-...
Comment · 4 years ago
Unfortunately we are not running the search concierge service indefinitely. But we will translate the findings into a search tool that makes it easier to answer complex questions. Sign up for our mailing list or shoot an email to [email protected] . We will let you know when its ready.
Comment · 4 years ago
In our sample, we found that people really cared about those topics but they were not urgent. Hence the reference to the Eisenhower Matrix.
The current state of search tools doesn't make it easy to answer complex questions so complex questions go unanswered. By lowering the activation energy to do an activity, the market for that activity increases. That doesn't mean that activity didn't ever matter.
Comment · 4 years ago
That's definitely one way to do it. Our approach is not to literally run a concierge search engine, but rather build up intuitions on how humans effectively find knowledge on the Web and then build tools to make that easier.
Is that the end-game of the startup you mentioned?
Comment · 4 years ago
You should consider just using the Matrix protocol and innovating on the UI/features side.
Comment · 4 years ago
> Real improved search efforts need to somehow change the content that's out there at the same time as they improve they experience
Yes, this is our key takeaway too. However, that doesn't translate to "google has a much deeper moat than most people realize". The current state of search doesn't engender much loyalty. While it would take a lot to ditch Google entirely, it wouldn't take much to start eating away at some small class of queries.
Comment · 4 years ago
Yep, graph structure of topics enables users to wander around topic space. Good for less directed, more exploratory searching.
Another graph that is useful is the graph of people -> topic clusters. See https://twitterverse.net/ . Such a graph can help rank content from people deeply invested in a particular topic, and its hard to fake because they would presumably have to trick all their peers about their expertise
We'll have our own Show HN soon but it's great to see similar ideas bouncing around. Would love to connect over email to learn more about your thoughts.
Check out our website: https://re-search.xyz Email us at [email protected]