Hi!
Will definitely add those to the docs and landing page — really appreciate the feedback! :)
We originally built Pivot for our internal observability pipeline, where we streamed the same data into both ClickHouse (with 1 month TTL - for fast dashboards and customer-facing analytics on recent data) and Iceberg (for longer data archiving and ETL jobs). We were frustrated with this duplication (and with ClickHouse OSS being disk-based rather than S3-based!), and got curious about how fast an Iceberg-native real-time analytics database could be!
Re DuckDB — it's definitely an amazing piece of technology, but it didn't provide the full "database" experience we needed: ingestion, efficient compaction, high concurrency, cluster deployment option, and a single endpoint for our backends and dashboards to query.
Ultimately, we managed to cut the cost of our customer-facing analytics and observability pipeline by around 75%, so we figured it might be useful to others as well! :)
Re hardware variation, we definitely optimized for cloud native deployments.
While we do have some optimizations and kernels for specific instruction sets on both x86 and ARM (for example AVX-512 on x86, and with Armv8.2-A as the minimum for ARM), most of our optimizations focus on taking advantage of the memory and network bandwidth of modern hardware, both of which have increased significantly in recent years (https://pivotlake.io/docs/database/why-pivot-is-fast/#utiliz...).
Will definitely add those to the docs and landing page — really appreciate the feedback! :)
We originally built Pivot for our internal observability pipeline, where we streamed the same data into both ClickHouse (with 1 month TTL - for fast dashboards and customer-facing analytics on recent data) and Iceberg (for longer data archiving and ETL jobs). We were frustrated with this duplication (and with ClickHouse OSS being disk-based rather than S3-based!), and got curious about how fast an Iceberg-native real-time analytics database could be!
Re DuckDB — it's definitely an amazing piece of technology, but it didn't provide the full "database" experience we needed: ingestion, efficient compaction, high concurrency, cluster deployment option, and a single endpoint for our backends and dashboards to query.
Ultimately, we managed to cut the cost of our customer-facing analytics and observability pipeline by around 75%, so we figured it might be useful to others as well! :)
Re hardware variation, we definitely optimized for cloud native deployments.
While we do have some optimizations and kernels for specific instruction sets on both x86 and ARM (for example AVX-512 on x86, and with Armv8.2-A as the minimum for ARM), most of our optimizations focus on taking advantage of the memory and network bandwidth of modern hardware, both of which have increased significantly in recent years (https://pivotlake.io/docs/database/why-pivot-is-fast/#utiliz...).