Comment

Comment by mediaman

Clickhouse has a more intentionally built ingestion system. Duckdb has concurrency limits so you can't have a writer and a reader on the same file if they're not the same process (multiple readers is fine).

But that's not too hard to work around. You can either have a single process that owns both writing and reading that file, or you can do a data lake where you post updates as parquet files into object storage, and duckdb handles the catalog. The Quack protocol also basically fixes this (though still in beta).

With Clickhouse, you can of course still have tenant separation, but you have to do it by managing users within Clickhouse that map to users/tenants of your main app, so that you can restrict SQL access by tenant to only their data store. Not a huge deal but I just like the Unix "it's just a file" simplicity of "Tenant A gets to run arbitrary SQL against their separate read-only, no-ATTACH duckdb file".

parent mediaman 2 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton