roboben

Its not Trino, its the catalog you use in the back.

With DuckDB and the queries we got, we had to provision such huge pods/nodes that we made it work but could never get to an economical viability. In the end, most of our customers had pods running 46 vCPUs / 360GB and asked for more. DuckDB is amazing, it really does everything right on a single node to make your query run successfully to the end almost always but it just doesn't work in our cloud env. Imagine when a query is running for 12h with this since and then another 10 from the same customer come in :)

Trino on the other hand has a much more convenient distributed query concept, so you can much more dynamically assign resources.

We tried Iceberg and it was too slow writing data is the short story.

>The fast OLAP stores just don't scale as well as going object store to a proper query engine like Trino

this exactly why we started with this project.

Author here. Good question. AFAIK you cannot use Quack for anything that looks remotely like distributed querying.

Author here. I agree. We spent most of the time to make it work at the scale of our large design partners and probably should have stepped back earlier. And yes, with an agent you can have a much larger scope hashed out faster than before.

Author here. Good point. Trino is not made for fast response times. We are using it for long running data modelling jobs or nightly ETLs.

Author here. We didn't leave Clickhouse. It still drives most of what Posthog offers. I missed that bit of context in my post I admit. Instead we are looking into a new stack we can give customers so they can run their warehousing workloads on it instead of the centralized CH.

Stateless CH on Iceberg sounds interesting, we looked into it but Delta/Iceberg had other downsides for us.

Author here. This is not about our internal warehouse but the warehouse product we offer to customers. This is still running on our centralized Clickhouse. Long-running data modelling queries and multi-tenancy is a challenge on Clickhouse.

Author here. We went past the POC stage to actually make real customer workloads work, which we did. Probably yes, we spent too much time not rethinking our approach but sunken cost fallacy so that's why.

Author here. I admit I kinda left out the context which is important here. We did not replace Clickhouse. Instead we want to move our warehouse product from Clickhouse to another stack which suits the long modeling jobs and multi-tenancy better. We learned through all our early design partners running huge workloads what we need to change.

As much as I'd love to claim victory, we are not there yet :)

[Author] Basically what we are doing with the help of Trino: a distributed query engine which can use as many resources as the query needs dynamically.

[Author] Basically yes, but we spun up pods on EKS. It was hard for the customer to estimate how much they might need and hard for us to know how much capacity to provide. Given our plans on pricing on CPUHours (or similar) the logic was kinda broken.

[Author] it was more like: we underestimated the problem and complexity of workloads our customers want to run. We did not rewrite anything but developing a new stack we can give to our customers doing warehousing. Agreed that this context is not really clear from the blog post.

Hi! Author here.

We engineered, then we measured and then we tried to make it work, probably for a too long of a time before we decided to completely scrap our current architecture and go back to the drawing board. Happy to answer more questions.