ahachete

If you want an OSS sharded Postgres, just use Citus. Feature-full, proven, mature, boring.

Yeah, and I don't see how that changes my question, which still holds: did you try to fix the first project, and report the issues, before doing a full rewrite?

I assume you have reported the issues / crashes to pgcopydb. Wasn't it easier to fix them rather than rewriting from scratch? I assume rewrite was heavily AI driven, and if so, probably AI could have fixed pgcopydb rather than rewriting.

Or am I missing something?

Same here. Stopped reading.

I feel ya. I'm always middle clicking twice, story and comments.

This shouldn't be a script (no offense, great job!). It should be built into the browsers!

AGPL and copyleft licenses are not restrictive, they provide forward (open source) guarantees. The only thing they "restrict" is the ability to remove freedoms. Therefore is not a restriction, is a guarantee. A guarantee that the project will endure as open source.

I wouldn't go as far as calling permissive licensing "restrictive" because they actually allow for reducing freedoms; but copyleft is definitely not restrictive at all, it's the opposite.

Having said that, I like and appreciate both kind of licenses and I have and will continue creating open source software using ones or the others.

> note that the local NVMe setup does have backups and WAL-archival to S3, which provides data-durability with RPO of 10s of seconds.

That's good, it provides a reasonable level of RPO, although I prefer just 0. But with this the RTO (or, equivalently, the downtime) is potentially quite large.

> Even with HA setup I expect performance difference to be similar across systems (may be slightly lesser)

I have done some benchmarks and I have observed a 24-27% performance penalty for semi-sync replication on instances using local NVMe. That's why I say this is non zero.

> In regards to default configs, that was intentional as default tuning is a differentiation across services and that needs to be measured. However we plan to add more configurability on postgres tuning in the future.

That'd be great.

My 2 cents:

> Each run lasts 10 minutes, long enough to move past warmup and capture stable throughput

But short enough not to show the effect of checkpoints (which is quite noticeable). It's not representing a realistic workload. I'd use at the very least 30mins with checkpoint_timeout=5mins so you get at least 6 checkpoint cycles.

> We report average TPS, average latency, P95 latency, and P99 latency

Unless I'm mistaken, only average TPS is reported, but not TPS over time. Again, it varies a lot (especially due to checkpoints!) and you need variation over time to understand patterns and determine SLAs.

> We tested two scale factors: 6849 (~100 GB) and 34247 (~500 GB).

I'd set them an order of magnitude apart (100GB and 1TB). 1TB+ is nowadays not unseen and therefore represents a step into the scale path that I believe should also be compared.

> While all services offer high availability, the underlying architectures differ. Some use standby replication, others use shared or distributed storage layers. Since we are focused on single-node compute and storage performance, we tested without HA enabled to isolate that.

So again, it's not realistic. The impact of HA in production is quite measurable in performance, especially if semi-sync is used.

But more importantly, you are comparing systems that are already redundant and highly available (e.g. Aurora, writing six copies of the data) with a single-instance one writing (if I'm not mistaken) to an ephemeral local NVMe. This is watermelons to grapes comparison (I assume HA CH clusters talk over the network for each confirmed transaction; but here they only go to local NVMe).

> Default Postgres configuration

I believe this is a mistake. Postgres must be tuned (at least to a reasonable degree) and try to homogenize the configuration across the tested systems. Otherwise you are not benchmarking Postgres, but rather the conservationism in the decisions of the default configurations of the managed services.

> We did not compare pricing.

So you leave the fun part out?

> Systems included

I miss a self-managed EC2 cluster (with local NVMe) for comparison.

TBH I don't think it's that straightforward, I see it more of a notable architectural change. At a very high level, this means:

* Adding a sharding function, as you say.

* Developing an external service for metadata (shard placement) or alternatively have that metadata in one place and replicate (consistently!) to every query router.

* Implementing functions/catalogs for the users to understand the placement and configure/alter it.

* Implementing shard migration / rebalancing capabilities, possibly using Postgres logical replication (plus notable automation).

Here's one idea if you follow this path, something that Citus doesn't have: make the sharding function pluggable and pick one by default which is well-known and available in many languages (e.g. xxhash). If you do so, and guarantee stability of those functions, they could be used externally (applications) to route queries / inserts especially to the appropriate shard. While it makes application more complex, it may allow (combined with access to the metadata service) for faster ingestion paths (this is often known as application assisted sharding), and its not exclusive of the query routers.

Edit: formatting