benwilson-512

We have been doing exactly this at CargoSense! We have gotten very far on just Postgres but as our BI and other analytics queries have gotten more complex and our scale of “business tables” grown 20x over the last 18 months it just wasn’t cutting it anymore.

The CDC to Clickhouse flow has been very fast and reliable, and then for us at least setting up some incremental materialized views that set up what we need for reporting has been perfect. Clickhouse’s storage efficiency has been mind boggling.

My wife works as an acoustical consultant at a global construction firm. The things you hear about factories, offices, and even hospitals is wild. Don’t get me wrong the construction world works very hard to avoid issues but I think we in software tend to hold other engineering disciplines up on a pedestal that doesn’t quite match the messiness of reality.

Hey folks! Author of Absinthe here, feel free to ask me any questions.

Hey vryotek, short version: stickiness is not necessary or important. It does use a websocket.

To elaborate a bit, while each LiveView client is indeed associated with a long lived process in the LiveView server, the Phoenix framework provides hooks and other lifecycle management tools to minimize the impact of disconnects from eg blue green server deploys. Most interestingly for example with forms, the "automatic form recovery" does a process where when the reconnect happens it delays re-rendering until the client side has had a chance to send the current form state back to the server and allow it to synchronize.

That's very cool. Presumably as well this could be opt out if you had specific operations (perhaps from a write-only API) that don't need to wait to do further writes?

In field metering situations you can have the timeline of each meter, when it records data points, and then you have the different timeline of the backend service, when it actually _receives_ data points.

Bonus points if the meters can send data out of order.

ARM support for the JIT is a great addition for any of us on an M1! I believe on Intel the JIT was improving compile times by 30-50%.

In practice we've seen it actually improve performance, because when fetching a data range for a device fewer actual rows have to be fetched from the disk. You pick certain columns (like device ID) that remain uncompressed and indexed for rapid querying, and then the actual value columns are compressed for a range of time.

We've got a few billion rows in TSDB, pretty happy with it so far. Our workload fits the OLTP workflow more than OLAP though, we're processing / analyzing individual data points from IoT devices as they come in, and then providing various visualizations. This tends to mean that we're doing lots of fetches to relatively small subsets of the data at a time, vs trying to compute summaries of large subsets.

Compression is seriously impressive, we see ~90% compression rate on our real world datasets. Having that data right next to our regular postgres tables and being able to operate on it all transactionally definitely simplifies our application logic.

Where I see a lot of folks run into issues with TimescaleDB is that it does require that your related data models hold on to relevant timestamps. If you want to query a hypertable efficiently, you always want to be able to specify the relevant time range so that it can ignore irrelevant chunks. This may mean that you need to put data_starts_at, data_ends_at columns on various other tables in your database to make sure you always know where you find your data. This is actually just fine though, because it also means you have an easy record of those min / max values on hand and don't need to hit the hypertable at all just to go "When did I last get data for this device".

I recently installed two MrCool DIY minisplits which worked out great. They come with pre-charged quick connect coolant lines so you can install them without the need for flushing coolant or soldering the lines.

If you're comfortable putting a hole in your house for the line and running power, it can really save a lot of money.

> Specially now we are using all the hotwire stuff.

Maybe I'm missing something, but doesn't this have precisely the same latency characteristics as live view?

I do agree that some of the "you don't need redis" and similar are overblown (particularly claims about tasks replacing background jobs), but I think there is truth to these claims in the following sense: Particularly in Rails, Redis gets used not just as a tool, but also to manage shortcomings in the language / runtime. Eg: Hotwire. Ruby doesn't have a good way to concurrently manage stateful connections, so it has to fallback to Redis. Ruby doesn't have a good way to manage inter-process or inter-node communication, so once again Redis steps in to fill that gap.

There's nothing wrong with using Redis, it's a fantastic tool. Using it as a state holder for an event loop through ends up feeling extremely convoluted after you've written a few GenServers (Phoenix channels / liveview).

The most immediate impact I'm noticing locally is a ~25-30% improvement in compile times from the JIT, which is always a delightful find in any release.

Synthetic benchmarks of the Elixir GraphQL implementation show a 64% improvement! Very excited to see how this plays out with real load, but this is very promising.

And even if you're young, if you find yourself waking up at night with your heart racing, excessive snoring, etc, get it checked out. I got a CPAP machine at 20, and it's hard to overstate the impact. You can even just straight up buy them these days and adjust the settings yourself, sometimes the insurance premium on the sleep study is as high as the cost of the machine.