tibbar

Although the simplest exact algorithm for TSP is O(n!), you can use dynamic programming to get the difficulty down to O(n^2 * 2^n), which is quite tractable at n=20.

However, sometimes you can construct exact solutions to much larger real-world data sets (tens of thousands of points) by exploiting structure in those graphs.

I used Hex at $PREVJOB and it was easily one of my favorite tools. It is maybe the one BI tool that I have felt truly attached to in my career

Ah, I was using "enterprise" in the colloquial tech/B2B sense as a company of ~1000 people or so. There are about 8,000 businesses of that size in the US, but they employ about half the private workforce. It appears to me that Cassandra adoption is very heavy within that market segment, for instance.

(Keep in mind that pushing relational databases to 10k writes/sec is not optimal! Well before you get to that point, there are much more cost-effective options.)

On to a few other nits...

> 10k writes a second is a problem that a very small number of systems have relative to all deployed web servers, the normal case is far closer to HN.

Hmm ... I would estimate that the majority of deployed servers today use a distributed database, operating out of clusters for e.g. Google, Meta, Amazon, Netflix, Microsoft, CDN operators, and so on.

And, of course, HackerNews is also using distributed databases, so it's not a great counterexample...

> Re HN using other systems for search and api, that’s more related to supporting companies the vc funded - Algolia

HN uses Google BigQuery to support a company they VC funded? Anyway, this is "no true Scotsman"-ing databases.

The issue is that data complexity is also exploding in the age of LLMs, so it still takes a lot of archaeology to get reliable (as opposed to plausible) numbers.

I think this dynamic also requires more data engineers, but I don't personally see the split in demand between those roles that you describe. Data analysts often have business or domain context that data engineers lack.

Perhaps we'll start rolling data analysis work into "operations".

Wait, data analysis/data science is still a thriving human job sector! It's just that you can do way more analysis, way faster, than you could before.

Postgres/MySQL and friends are incredibly good at handling the kind of workloads they were designed for (transactional reads and writes, and perhaps some aggregate queries if you can plan your indexes in advance.) You can scale that quite far, although if you get big enough you will eventually run out of room there (around 10k writes/sec or so.)

However, relational databases are a poor fit for writing logs, for performing analytical queries, and various other purposes that most reasonably large businesses today need. You can try to use your relational DB for them, but you will create real performance issues very fast; and even if you can get the DB to scale up for the load, you'll end up paying way more money that you would from using the right kind of DB for that workload.

The killer app for UUIDv7 is logs. High-volume, distributed, non-public data that you probably want to partition by time as well. Log volume typically grows much, much faster than transactional writes, and any reasonably big organization is going to want to keep that data out of their relational database.

As far as relational databases go, another common workload they're a bad fit for is analytical queries. If you employ any data analysts or operations staff, they can bring even the biggest deployment of Postgres/MySQL to its knees pretty fast with relatively simple queries. Read replicas buy you a little runway there, but they're just not a good or cost-effective way to power BI workloads.

> To take one example this high traffic website (HN) uses one db

First of all, HN does not use a single database! E.g. it powers search with Algolia and API queries via Big Query, and that's just for external traffic.

Second, HN is not a high (write) traffic website. It's a forum. We can actually see from those serial IDs that there are about ~50 million all time comments. HN is about 20 years old; that works out to about 5 writes per minute. Of course growth has increased over time, but it's still probably less than 1 write per second.

So, I'd agree with you that lots of interesting websites can power their primary application write traffic with a traditional relational database, but I think you'll find that those properties end up needing several other databases anyway for other purposes.

This is actually a very interesting question. I assume that you are referencing extensions that add shards to Postgres, since it runs on a single server out of the box. You made me curious how those extensions/rewrites handle sequences.

It looks like Citus (one such extension that runs shards on separate workers) does something a little like what you described. Each node gets an ID, which becomes the high bits of the sequences it generates. So serial IDs are monotonic within a shard, but not across shards.

Aurora PostgreSQL Limitless uses different abstractions, but also ends up reserving different chunks of IDs for different routers to use.

Others have mentioned their difficulty in distributed systems; but even for traditional enterprise workloads, serial IDs don't scale very well. They force all writes through a single, transactional bottleneck, so you can't shard your data. UUIDv7 and friends lets you guarantee that records get a unique identifier without that coordination cost. There are also scenarios where you need to have access to an ID before the write to the database succeeds.

Of course, lot of event/log data never hits a relational database at a all.. it gets piped right to a data lake. But those records still need IDs, too.

Many years ago, I was working at a small shop that used Oracle databases. We had different tenants for different customers. However, we still wanted records across different tenants to have unique IDs, so (as I recall) all writes to all tenants phoned out to a central sequence generating function. This became a major source of latency for us. This is jut one of many ways that serial IDs become annoying if you're doing anything other than running a 1990's-style RBDMS on a single server.

Anyway, once you try to solve this problem, there are lots of schemes that give you (almost certainly) unique IDs without coordination. Whether you pick UUIDv4 or v7 etc.etc depends on other factors. But one nice feature of UUIDv7 is that it is kinda, sorta serial, which is helpful for e.g. partitioning to work well.

This is such a conspiratorial mindset. This very forum thread is packed with people providing code and confirmation of working exploits. Leading security organizations have confirmed RCE exploits with video and provided detailed writeups of how the bug works. The PHP maintainers coordinated the release. The government is involved.