bonobocop

The durability and transformation reasons are definitely more compelling, but the article doesn’t mention those reasons.

It’s mainly focused on the insert batching which is why I was drawing attention to async_insert.

I think it’s worth highlighting the incremental transformation that CH can do via the materialised views too. That can often replace the need for a full blown streaming transformation pipelines too.

IMO, I think you can get a surprising distance with “just” a ClickHouse instance these days. I’d definitely be interested in articles that talk about where that threshold is no longer met!

Sure, but the article doesn’t talk about that, it seemed to be focused on CH alone, in which case async insert is much fewer technical tokens.

If you need to ensure that you have super durable writes, you can consider, but I really think it’s not something you need to reach for at first glance

Yeah, handles all the OTel signals

Not OP, but to me, this reads fairly similar to how ClickHouse can be set up, with Bloom filters, MinMax indexes, etc.

A way to “handle” partial substrings is to break up your input data into tokens (like substrings split in spaces or dashes) and then you can break up your search string up in the same way.

Claiming delay compensation if you don’t have intent to travel is the fraud part.

Easiest example is if you have a season ticket, but you have the day off. You weren’t going to take the train to work that day, so no intent to travel. If you claim DR, then that’s fraud for the compensation.

I don’t think it’s fraud on the DR side if you actually take the trains and intend to travel.

If you didn’t actually intend to travel, then claiming DR is fraud.

The DR system doesn’t look at ticket scans alone. It also builds a profile per customer based on a number data points.

It will flag up quite quickly if you are “sniping” delayed trains at different times.

A small question on the schema, I noticed that you have only “_now” as the Order By (so should just use that for the primary key). Do you expect any cross tenant queries?

Just my feeling would be that I’d add the tenant ID before the timestamp as it should filter the parts more effectively

Thoughts on stuff like ClickHouse with JSON column support? Less upfront knowledge of columns needed.