cfors

The spirit of this is correct, but a better approach to this is going for a walk with just your thoughts.

Yes, that means no phone, no headphones, just you and your brain enjoying a walk. Let your mind wonder and be free.

The underlying C library interacts directly with the postgres query parser (therefore, Postgres source). So unless you rewrite postgres in Rust, you wouldn't be able to do that.

Just curious what the state of the art around filtered vector search results is? I took a quick look at the SPFresh paper and didn't see it specifically address filtering.

In any API service, it's better to handle via dependency injection IMO.

Instantiate all of your metadata once, and then send that logger down, so that anybody who uses that logger is guaranteed to have the right metadata... the time to add logging is not when you are debugging.

Might be missing my argument here - I stated that there are workable solutions to this like you have pointed out.

But ANN search is still a sledgehammer and building out hybrid solutions that help bridge the gap between this and traditional data stores still have room for innovation.

Sure they can handle the basic case of ANN. But ANN still doesn’t have good stories for lots of real-world problems.

* filterable ANN, decomposes into prefiltering or postfiltering.

* dynamic updates and versioning is still very difficult

* slow building of graph indexes

* adding other signals into the search, such as query time boosting for recent docs.

I don’t disagree these systems can work but innovation is still necessary. We are not in a “data stores are solved” world.

I don't disagree that rock solid is a good choice, but there is a ton of innovation necessary for data stores.

Especially in the context of embedding search, which this article is also trying to do. We need database that can efficiently store/query high-dimensional embeddings, and handle the nuance of real-world applications as well such as filtered-ANN. There is a ton of innovation in this space and it's crucial to powering the next generation architectures of just about every company out there. At this point, data-stores are becoming a bottleneck for serving embedding search and I cannot understate that advancements in this are extremely important for enabling these solutions. This is why there is an explosion of vector-databases right now.

This article is a great example of where the actual data-providers are not providing the solutions companies need right now, and there is so much room for improvement in this space.

The README is 100% just autogenerated by Claude. It looks like every README generated from these tools.

Can’t speak for the code since I haven’t peaked into it

Just wanted to say thank you for this article - I've read and shared this a few times over the years!

When Breath Becomes Air by Paul Kalanthi.

A fascinating memoir by a philosopher turned brain surgeon, facing a terminal cancer diagnosis. A person who spent their entire life pondering the morality of life being faced with their own ultimatum.

I reread it once a year, at minimum. A deeply moving book.

My system’s programming teacher spent an entire week of a 12 week semester (3 classes) going over how to use gdb/vim for one of our projects.

Still my favorite class and professor from my time at school.

Can anybody speak to how Vespa compares to some other Vector Database solutions? Seems like there's so many options today