Rietty

Why were the HFT firms suing employees?

And this is why I just love Airflow so much.

Some of the largest are a few billion rows and we sample randomly when developing code then execute it on all

Many of my datasets are similiar!

Depends on the dataset anywhere from seconds to tens of minutes depending on preprocessing needed.

Working in a Data Engineering/Operations role which focuses heavily on financial datasets. Everything is within AWS and Snowflake and each table can easily have >100M records of any type of random data (there is a lot of breadth.) General day to day is creating jobs that will process large amounts of input data and storing them into Snowflake, sending out tons of automated reports and emails to decision makers as well as gathering more data from the web.

All of this is done in a Python environment with usage of Rust for speeding up critical code/computations. (The rust code is delivered as Python modules.)

The work is interesting and different challenges arise when having to process and compute datasets that are updated with 10s of TBs of fresh data daily.

According to a friend I know who lives in the tri-state area that is what happens to them, but they max out 401K, have insurance etc.

What if the LLM gives them bad information and they don't know it? I personally would also just ask in a thread than risk the LLM info.

> Yet the super fast probe with apparently made it not an issue?

Could you explain how that makes it a non-issue? It seems counter-intuitive to me that it solves the problem by just probing faster?

> we had a hack to drop all messages in a process's mailbox through the introspection facilities and sometimes we automated that with cron...

What happens to the messages? Do they get processed at a slower rate or on a subsystem that works in the background without having more messages being constantly added? Or do you just nuke them out of orbit and not care? That doesn't seem like a good idea to me since loss of information. Would love to know more about this!

Does striking mean you are unemployed, and if so, why? I mean I understand that you're not gonna get paid if you strike.. but feels weird to also have other things affected? Then again I don't know how the US works. (Genuine question)

Minecraft is not an online game, it has a multi-player option sure.. but it doesn't strictly need to be online. There is a single-player mode since basically forever..

It's one of the wealthier neighbourhoods afaik, so apparently it's pretty nice. I think 'low 80s' means when they started renting there.

I understand the first two, but I could never wrap my head around the "having an SO" is a scandal. Why is that?

You don't AFAIK. Mine just asked for a routing and account number.