Comment

Comment by jordepic

TL;DR: IceStream keeps a secondary index of iceberg data on object store to enable efficient conversions from equality deletes to positional deletes in a horizontally scalable manner. After benchmarking this a bit you can see order of magnitude improvements on large tables, which is hugely important for streaming pipelines!
parent jordepic 4 months ago

No comments yet

Be the first to share your thoughts.

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton