Comment

Comment by pdet

The main reason was due to a much easier tight-integration with the DuckDB internals, so we can take the most of the DuckDB engine (and existing code) to use. As the extension must interact with table scanners, casting functions and whatnot.

However, DuckLake doesn't need to be implemented in C++ at all, I believe that whatever fits the engine that will be performing the reads/writes in Parquet and the connectors for the DBMSs should be a good fit.

For example https://github.com/borchero/ducklake-sdk - Rust https://github.com/datafusion-contrib/datafusion-ducklake - Rust https://github.com/motherduckdb/ducklake-spark - Scala https://github.com/brikk/trino-ducklake - Kotlin

Disclaimer: I'm the lead developer of DuckLake.

parent pdet 22 days ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton