luuio

Happy to continue :)

Where our conversation is at, it's not about RDBMS vs. NoSQL.

On the Redis INCR operation example (or an equivalent operation in any database, even in an RDBMS). In short, you don't always have the guarantee to know the result of the operation. This is part of what the 99.9% means when services advertise their reliability. 99.9% of the time you'll get an acknowledgement back, but not 100%.

The question comes to, what happens if I miss an INCR, or if I accidentally double INCR when I retry? For some use cases, it's acceptable: like counting the number of views on a Youtube video. For others, say a financial system counting money or making a payment, that's not OK. That's where idempotency really matters (btw this is why many redis operations have the NX option).

Let's go completely off track for a little bit, say you're Strava. You let devices and users upload workouts to your service. What happens when a phone sent you the workout, then disconnected due to bad receptions? In this case, the phone doesn't know if the operation succeeded so it's forced to retry. How do we make sure the workout isn't created and counted twice on Strava? Well, each workout has a unique UUID generated by the device. Strava uses this UUID to dedup when it receives an upload. The UUID is the idempotency "checkpoint".

In this case, it doesn't matter what kind of database Strava is using. The idempotency problem is the same.

Look up the two general problem on youtube. Unless the entire end to end operation is wrapped inside a giant transaction, no system in the world can give you the confirmation.

Imagine this: you issued a write, a few things can happen: 1. The callsite crashed, maybe due to an out of memory issue or whatever. You don't know if it succeeded

2. The database returned an acknowledgement, but then the callsite crashed before storing the acknowlwedgement for the next step.

let's say you have an object like this when you started: {count: 10, etag: 1}. then for some reason, something failed.

when you retry and load the object, you get {count: 12, etag: 3}. how do you know if your previous attempt had successfully persisted or not, or if the updates to the object came from other processes/requests?

you're mixing up conflict handling vs. idempotency

> why wouldn't doing the same write twice produce an idempotent result

you can imagine this:

```

var thing = KVStore.Get<MyThing>(...);

if (things.Checkpoints.Contains(myUuid) == false) {

  thing.Counter += 1;

}

KVStore.Update(thing); ```

having an etag doesn't help you with retries, where we expect that `thing` could be mutated by another flow between your retries.

> there is always more wisdom in what we throw away than what we innovate

i have found this to be true throughout my career. not that things cannot improve, they can always improve. but assuming the original work wasn't grossly incompetent, anything "new" tend to make the same [undocumented] mistakes

1. Beyond just querying, the stored proc spent a lot of time processing data. As in, looping through cursors, making business logic decisions, calculating things, etc.

2. Having the business logic (not just loading the data) inside the stored procs meant that a change in business logic that would normally only need to update application code, now invalidates the stored procedure's cached execution plan.

The KV store had etag support for conditional writes. Etags are only useful to make sure the data didn't change underneath between your read and your write.

Storing the checkpoints along with the mutation was for idempotency. If the checkpoint was in the document, that meant the mutation had succeeded and a retry should be no-op

Not Dan Luu. And to the other comment, not intentionally hiding my identity. Just that for most of my writing, who I am is irrelevant.

Windows licensing cost. They are a pretty penny at large scale.

a random number (existing entropy) XOR with a static number (the crashed wall) is still a random number, me think

Looking through that list, Windows XP logo has the perfect balance of:

- Simple solid modern colors, no halo/vignette effect like the Vista/7 logos

- Multiple colors, not just one single boring blue color like the newer logos

- The S-curve lines are just soft enough, yet not overly complicated, but also not boring straight lines