menthe

First of all, what are you optimizing for? Expiry latency shouldn't matter, only keeping pace with insert throughput. And while I appreciate that you sidestepped the immediate issue through the solutions presented in the article, dropping the foreign key and adding an anti-join doesn't feel like a long-term solution, since you trade referential integrity for orphan risk.

There's a point in PostgreSQL scalability where the need to vacuum (following updates/deletes) becomes a problem in itself, and that point is a lot closer than most anticipate. For instance, vacuum can’t be unable to reclaim versions unexpectedly (e.g stuck transaction holding the TID horizon), leaving you with millions of dead tuples & massive stale index linked-lists, which can lead to major service degradations.

An alternative could be to tombstone rows with a HOT UPDATE on a `deleted_at` column. If that update is HOT (heap-only), the new row version lives on the same page and no new index entries are created. The old version does become a dead tuple, but it's pruned in-page and ad-hoc by ordinary reads and writes, so VACUUM is never needed to reclaim them, and dead tuples never accumulate, least of all in the indexes. On PG16+ this even works with a BRIN index on `deleted_at`, given some fillfactor headroom.

That also addresses the 20-minute scan, which seems to have been caused by walking millions of dead index entries at the oldest end of created_at, each needing a heap visit to confirm it was dead. With HOT tombstoning, those dead entries never get created in the first place. A persisted keyset cursor (the last processed `created_at`) would avoid re-walking that region either way.

The manual VACUUM also couples your business logic to the storage layer, which is a bit of a slippery slope that’ll cause issues when you scale to the next order of magnitude.

I understand why long-running workflows and per-class retention complicate partitions. But if most workflows are short-lived, partitioning plus a tombstone sweep for the long-lived stragglers could avoid most DELETEs, and would also help index sanity, caching.. and let you profit from the many other benefits of partitions.

Seems everyone on Hacker News has already forgotten about Anthropic’s amazing new watermarking algorithm. You don’t need to look any further to understand why it writes so awfully - it literally swaps words programmatically all over the text from the result of a hash function…

Unfortunately - the widespread propaganda seen around both nuclear power plants and datacenters is ultimately going to cost America everything - the American way of life our child’s will inherit is going to mean something very different than what it meant for us.

China controls 90-99% of the rare earths - which are necessary for everything from cars to defense, chips, energy and even construction steel;

China controls 70%+ of critical minerals (99% of galium/magnesium, 85% tungsten, etc, etc).

China owns the world’s manufacturing - with the associated skilled tradesmen (hard to produce anything fully locally - the experts are retired / passed already).

China has abundance of power and energy.

China has got both power, energy, and they’ve got the will.

China has got the best intelligence models today (as of this month) - which means scientific superiority, increased productivity, and military superiority.

What does America have, and what does that mean for our economy in the coming 20-40 years?

The reality is that we are in a new race to intelligence and re-industrialization, but we seem to be anti-everything at this point. Nobody wants to hear it, I know I am going to get downvoted to hell with this, but this is ultimately the new reality, and our eternal debating / 4-years cycles vs. life-long strategies in China, is going to cost America everything.

Nothing a well-trained model won't be able to instantly solve. It's literally just grunt work, not rocket science.

F DRMs though. Good news is those AAA games are rarely worth anyone's time anyways. Better spin up indies or classic games - a good SNES game is worth a hundred of those garbo AAA license rehashes.

Can we just flag this kind of misleading and propagandist content? It’s literally content+karma farming, and brainwashing hot garbage.

Total rage bait that doesn’t have its place on platforms catering for higher educated and science oriented people.

Nuclear plants evaporates water, water rains down a few kilometers away, gets collected in sewer, recycled/filtered and re-injected in the system.

You know what doesn’t get recycled? Coal & gas burning pollution & CO2. And you know what doesn't get re-injected? Lost lives to pollution and global warming.

Take this to daddy Elon - this has absolutely nothing to do with Rivian.

Meh, typical overblown “take” ihmo.

> All this stuff is just done automatically in Rust. If you have to write out the code by yourself, that's more work, and a huge chance for bugs. The compiler is doing some very complex tracking to figure out what to drop when and where. You'd have to do that all in your head correctly without its help.

What prevents anyone from dedicating a Zig memory allocator to the job (and all of its subtasks), and simply freeing the entire allocator at the end of the job? No baby-sitting needed.

Or if the mindset is really to be assisted, because “very complex” and too “much work”, may as well use a garbage collected language.

> It's knowing the compiler is on your side and taking care of all this that makes it magical.

Until you got used to it, and trusted it so much, and it suddenly misses something - either after a compiler update, or after some unsupported code introduced, and that shit takes down prod on a Friday. I’m not going to take the chance, thank you, I can call free() and valgrind.

I am sorry, are you non-sarcastically arguing that being able to pass through airport security, potentially accessing cockpits and planting bombs onboard airplanes, with a high-school level SQL injection on a federal website used by dozens of airlines & airlines employees, is actually, "fine"?

Besides, I am not sure what sort of "security through obscurity" you are talking about? Ian and Sam found it, and frankly - with a public page, page title + first h1 tag clearly stating that this relates to a Cockpit Access system, this has got to show up in a shit ton of security research search engines instantly.

Not 100% sure why it’s often idolized on HN.

We’ve maintained a financial exchange w/ margining for 8 years with it, and I guarantee you that everyone was more than relieved - customers and employees alike, once we were able to lift and shift the whole thing to Java.

The readability and scalability is abysmal as soon as you move on from a quant desk scenario (which everyone agrees, it is more than amazing at.. panda and dask frames all feel like kindergarten toys compared), the disaster recovery options are basically bound to having distributed storage - which are by the way “too slow” for any real KDB application given the whole KDB concept marries storage and compute in a single thread.. and use-cases of data historical data, such as mentioned in the article, become very quickly awful: one kdb process handles one request at once, so you end up having to deploy & maintain hundreds of RDB keeping the last hour in memory, HDBs with the actual historical data, pausing for hourly write downs of the data, mirroring trees replicating the data using IPC over TCP from the matching engine down to the RDBs/HDBs, recon jobs to verify that the data across all the hosts.. Not to mention that such a TCP-IPC distribution tree with single threaded applications means that any single replica stuck down the line (e.g. big query, or too slow to restart) will typically lead to a complete lockup - all the way to the matching engine - so then you need to start writing logic for circuit breakers to trip both the distribution & the querying (nothing out of the box). And then at some point you need to start implementing custom sharding mechanisms for both distribution & querying (nothing out of the box once again..!) across the hundreds of processes and dozens of servers (which has implications with the circuit breakers) because replicating the whole KDB dataset across dozens of servers (to scale the requests/sec you can factually serve in a reasonable timeframe) get absolutely batshit crazy expensive.

And this is the architecture as designed and recommended by the KX consultants that you end up having to hire to “scale” to service nothing but a few billions dollars in daily leveraged trades.

Everything we have is now in Java - all financial/mathematical logic ported over 1:1 with no changes in data schema (neither in house neither for customers), uses disruptors, convenient chronicle/aeron queues that we can replay anytime (recovery, certifying, troubleshooting, rollback, benchmarks, etc), and infinitely scalable and sharded s3/trino/scylladb for historical.. Performance is orders of magnitude up (despite the thousands of hours micro-optimizing the KDB stack + the millions in KX consultants - and without any Java optimizations really), incidents became essentially non-existent overnight, and the payroll + infra bills got also divided by a very meaningful factor :]

"without ever having a job"

First Google Result for "nofx retire":

> The punk group have all got normal jobs, which they will continue to pursue, but are pulling out all the stops for one final tour for their army of fans.

Seems in direct contradiction with this anarchist click bait title.

I don’t get it. If you do not want to participate in the preservation and distribution of the archive, why don’t you just move on instead of complaining?

Besides, gluetun+chihaya+qbit containers do the job without breaking a sweat, and without ever having to remember that you run a VPN - as it’d only be tunneling the containers of your choice. gluetun is the best image ever made!

Everyone’s talking about how you MUST hit the database for revocations / invalidations, and how it may defeat the purpose.

How is no one thinking of a mere pub-sub topic? Set the TTL on the topic to whatever your max JWT TTL is, make your applications subscribe to the beginning of the topic upon startup, problem solved.

You need to load up the certificates from configuration to verify the signatures anyways, it doesn’t cost any more to load up a Kafka consumer writing to a tiny map.

Thank you very much for having taken the time.. Your comments and function are both very helpful!