citguru
Joined 7 years ago
Karma
102
Submissions
42
Member since
Nov 10
Recent submissions
Comment · 3 months ago
Comment · 6 months ago
I needed to back up terabytes of Postgres RDS data to R2 without intermediate local dumps or AWS egress fees, I could not find any ideal solution, so I built this.
Comment · 6 months ago
Hi,
DuckLake is great for the lakehouse layer and it's what we use in production. But there's a gap and thats what I'm trying to address with OpenDuck. DuckLake do solve concurrent access at the lakehouse/catalog level and table management.
But the moment you need to fall back to DuckDB's own compute for things DuckLake doesn't support yet, you're back to a single .duckdb file with exclusive locking. One process writes, nobody else reads.
OpenDuck sits at a different layer. It intercepts DuckDB's file I/O and replaces it with a differential storage engine which is append-only layers with snapshot isolation.
Comment · 6 months ago
The project is still fairly new and not close to production tbh.
I'd actually recommend the simplest option: just write them to Parquet on S3 and query with plain DuckDB. Or you could use Ducklake - https://ducklake.select/
Comment · 6 months ago
Thanks for this, really enjoyed reading this and helps validate some of my personal thoughts
LaunchPort: launch your side project and get real customers without incorporating first. You'd keep the code, IP, and customers. Infra, payments, sales/marketing and legal handled for you; you "graduate" into your own company whenever you're ready.
Do you get what this does? Would you actually sign up? What would stop you?
Landing page for context: https://launchport.lovable.app/