nikhilgarg28
Joined 14 years ago
Karma
268
Submissions
23
Member since
Mar 3
Recent submissions
Comment · 3 years ago
One of the authors of the post here. I've never heard of this approach but it sounds really cool. Will definitely try it out someday!
Comment · 3 years ago
We hadn't considered that at all, will definitely check it out and explore further. Thanks!
Comment · 3 years ago
Yes, it wasn't initially there mostly due to an oversight but we later edited the article.
Comment · 3 years ago
What is the size and dimensionality of your corpus?
Comment · 3 years ago
The article isn't meant to present a new library competing with Annoy or other state of the art approaches, but instead, describe how vector search works using short code snippets.
I've now updated the article to clearly mention this and also cite annoy.
Comment · 3 years ago
We did some benchmarking and found that an index of 1M vectors using 15 trees can be built up in ~115 seconds (8 CPU, 8 GB RAM). https://github.com/fennel-ai/fann/pull/8.
Given this, around a second to build an index of 10K items sounds about right.
Comment · 3 years ago
Apologies, that got omitted by mistake. I've now included the post to include that piece of code as well.
Comment · 3 years ago
One of the authors of the post here. Happy to take any questions!
Comment · 3 years ago
One of the post authors here. While building/writing this was fun, FWIW, we were really surprised by how fast the Rust version turned out to be with relatively little effort.
Comment · 3 years ago
Fennel | Distributed System Engineer | Menlo Park (Bay Area) or Remote | Full time (https://fennel.ai/)
Fennel is an ML Infra company building realtime feature engineering platform. As part of this, we are building a streaming database with few ML specific layers of abstractions on top.
We are looking for a senior distributed system engineer. Preferable (but not necessary) to have prior experience in database internals and/or stream processing systems. Bonus if you have written Rust before.