Comment

Comment by electroly

It was for work so I can't share it, but it's read-only and assumes an immutable database which puts it into the realm of a weekend project. I generate the database files in a batch process with regular SQLite, then upload it to S3 and query it using the S3 VFS. If you pull up the S3 API and the SQLite VFS API it's pretty straightforward to see how to glue them together for a simple read-only VFS. I do byte range requests to pull out the pages that SQLite requests, and a readahead cache helps reduce the number of requests.

There are some open source codebases that do similar things, but take it all the way with write support: https://github.com/uktrade/sqlite-s3vfs

parent electroly 1 year ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton