Comment Comment by esafak If I understand correctly, polars relies on delta-rs for Delta Lake support, and that is what does not support Deletion vectors: https://github.com/delta-io/delta-rs/issues/1094It seems like these single-node libraries can process a terabyte on a typical machine, and you'd have have over 10TB before moving to Spark. parent esafak 1 year ago
It seems like these single-node libraries can process a terabyte on a typical machine, and you'd have have over 10TB before moving to Spark.