llogiq

The problem with C++ vs. unsafety is that there is really no boundary: All code is by default unsafe. You will need to go to great lengths to make it all somewhat safe, and then to even greater lengths to ensure any libraries you use won't undermine your safety.

In Rust, if you have unsafe code, the onus is on you to ensure its soundness at the module level. And yes, that's harder than writing the corresponding C++, but it makes the safe code using that abstraction a lot easier to reason about. And if you don't have unsafe code (which is possible for a lot of problems), you won't need to worry about UB at all. Imagine never needing to keep all the object lifetimes in your head because the compiler does it for you.

So professional work during a worldwide pandemic with all the hardship that entailed suffered...from working remotely? I don't buy that.

That very much depends on the code you're compiling. The factors that come into play are monomorphization (which means the compiler builds one copy of the code per type it is called for), procedural macros (which need to be fully compiled before being able to expand code using them), whether complex type shenanigans are used, etc. etc. Absent that, Rust will compile roughly as fast as C nowadays.

Very nice diagrams, they make the article really easy to follow! This really drives the point home that vector search isn't only a qantitative (as in faster), but a qualitative evolutionary step.

Makes one wonder what other use cases are lurking that would need just another small modification and haven't even been thought of yet because they used to be impossible to implement.

I totally agree that latency of this solution leaves a lot of room to improvement. But that's totally besides the point of the article, which is that people can get a no-cost semantic search for their personal website using those services. They can also use other solutions, of course.

Also I'm experimenting in further integrating things to reduce latency and most likely will publish another article within the month. Stay tuned.

Finally I somewhat agree that many of the players in the vector DB space try to push their cloud offerings. Which is fine, how else should they make money? And if latency matters that much to you, Qdrant offers custom deployments, too. I believe running Qdrant locally will handily beat your LanceDB solution perf-wise unless you're talking about less than 100k entries. We have both docker containers and release binaries for all major OSes, why not give it a try?

The initial version of this actually used Mighty, but I didn't find any free tier available, so I switched to Cohere to keep the $0 pricetag.

I also found while writing the article but after I had already done my research that cargo-lambda has grown some additional functionality that could have removed the need for the AWS CLI, but I wanted to get the article out, so I didn't test-drive that.

Thank you for the suggestion! I actually thought about using terraform, but I wanted to keep the experiment somewhat minimal regarding technologies and as I had already added AWS Lambda and Rust to the tech stack, I wanted to stay as close to the metal as possible. Besides, this is not for commercial applications, so I don't think high availability is in scope.

When starting this project I thought the same thing, but having done it I honestly cannot tell that much of a difference. Yes, there are two more steps in setting up the Lambda function, but in the end you still write an HTTP server and have them serve it.

Author here. This was a fun exercise to produce a semantic search using only free-tier services (which in the case of Cohere means non-commercial use only and in the case of AWS Lambda is limited to one year).

It also marks my first foray into using cloud services for a project. I've long been a cloud sceptic, and doing this confirmed some of my suspicions regarding complexity (mostly the administrative part around roles and URLs), while the coding part itself was a blast.

Nick's articles are always a delight to read, and this one is no exception. Openly discussing failure is one part, but coming up with so many ideas and having the gumption to actually try them out on a compiler is really fascinating.

This is a nice article for people new to async/await in Rust.

Well, if the programmer writes the type annotation to actually have their code checked, how would a gradually typed language differ in practice from a static language?

Yes, you can still do dynamic typing. But I'd argue that using `dyn Any` you can do so in Rust, which is a statically typed language if I ever saw one.

Otherwise I completely agree about the languages growing closer together.

Yeah, that one hasn't aged too well. We've all seen the backswing to statically typed languages. Yes, some of them (e.g. typescript) run on top of dynamic langs, or allow for VMs (hi, WASM!). Why? Because ironically the same famed flexibility that makes it oh so easy to whip up a prototype is biting us in the ass when it comes to make a production-grade piece of software while staying on top of the ever changing requirements. So while we have a lot of python in ML (where most things haven't left prototyping stage), a lot of code nowadays is written in languages like Rust, TypeScript, Swift and others.

Not (only) because that's faster to run, but because it's faster to change while still remaining somewhat working correctly. And the current crop of compilers not only can produce stunningly fast code, but also awe-inspiringly great error messages that put the 90's and oughties' cryptic error messages to shame. Try that with a dynamic language!