vadman97

If there was only one alert criteria, that'd be simple. Our alerts can be configured for any data filters (eg. only matching logs with column `level='error'`); we would have to create a unique MV for each alerts' filter condition.

Forget the exact details, but we ran into issues with tuning the two setting you're describing on the Cloud instance (these settings were user level settings that seemed to get reset by ClickHouse Cloud because they were updated based on the cluster size). Perhaps this could change with Cloud and isn't an issue with the OSS version.

Once we hit >100k inserts per second, async inserts didn't work well for us because we had limited control over the background async insert batching happening on the cluster. The background inserts would be too small, resulting in many merges running, causing high CPU, causing high back-pressure (latency) on async inserts which would just result in an ingestion delay.

Plus, async inserts are only available on ClickHouse Cloud.

Materialized views can help change the ORDER BY with 0 downtime:

* Create a new version of the table with the new ORDER BY.

* Create a materialized view that will insert from the old table to the new table.

* Update your application code to query the new table.

* Start inserting data into the new table.

Exploring bloom filter index merges would be an interesting addition. I do wish it were easier to profile merge performance to break down where most of the CPU time is being spent.

3 nodes of 120 GiB and 30 vCPU each.

I see what you did there.

Our browser client would help with tracing what network requests are being made by said apps since it would capture all network requests. However, Shopify may restrict what our browser client can do in your frontend store (I'm guessing that the 3rd party apps are added to a shop as iframes or are otherwise sandboxed).