Comment by choppaface

Not a full solution, but seeing the OP seeks to be a key-value store (versus full RDBMS? despite the comparisons with Spanner and Postgres?), important to weigh how Rockset (also mainly KV store) dealt with S3-backed caching at scale:

  * https://rockset.com/blog/separate-compute-storage-rocksdb/

  * https://github.com/rockset/rocksdb-cloud

Keep in mind Rockset is definitely a bit biased towards vector search use cases.