smarvin2

It does work by running the LLM on the database server but you can configure the LLM to run on the GPU

You can use Korvus for search and feed the results to an external model

Great question! Making calls to external services is not something we plan to support. The point of Korvus is to write SQL queries that take advantage of the pgml and pgvector extensions. Making calls to external services is something that could be done by users after retrieval.

Unfortunately the pgml extension does not work on AWS RDS so there is not.

You can totally use an existing postgres database. Just make sure to install the pgvector and pgml postgres extensions and it will work!

We would love help developing a Ruby SDK! We programmatically generate our Python, JavaScript, and C bindings from our Rust library. Check out the rust-bridge folder for more info on how we do that.

I think there are a couple things worth noting here:

Semantic search performs well at capturing documents keyword search misses. As noted in the article, when searching for exact keywords, keyword will outperform semantic search. It is when users do not know the exact phrase they are looking for that keyword search shines.

Semantic search should only be a part of your search system, not your entire search system. We find that combining keyword search + semantic search and then using a reranker gives the best results. It is best if the reranker is fine tuned on your search history, but general crossencoders perform surprisingly well.

Hey thanks for bringing this up. I'm not able to reproduce the error you are getting. I just ran `cargo install lsp-ai -F llama_cpp -F metal` on my mac and received no errors and a clean install. Can give me the full output of your command either here or in a github issue?