antonio2368

Sadly I never put enough effort into trying out such checks. but your excitement about them gives me motivation to properly try them out.

I didn't think about the vDSO, and that it's used for `clock_gettime`, so thank you for the information and correction!

It's really hard for me to get a feel of what is okay to use, what is the standard way to do something without actually implementing everything and seeing how it behaves, so information like this help me a lot.

> __rdtsc drift isn’t an AMD bug, it’s by design.

Yes, I should've emphasized this.

> hey probably just got lucky in that the drift always exceeded the time that elapsed between calls on other cpus

And this is also true. :D But I would say we were lucky to notice this problem at all because it really doesn't happen on that many CPUs, at least on the ones we tried out.

Sorry if something was explained awkwardly or incorrectly, I'm trying to get a hang of writing posts like this, technical posts that are interesting and fun to read.

> If they were using unsigned arithmetic then they would've gotten the expected value.

And we did, but we did not expect to get the input values we got, which are explained by your last paragraph: "time stamp counter drift across cores." This was the real problem, and something new we learned.

> use clock_gettime+CLOCK_MONOTONIC

I forgot to include this because I was focused on the solutions we ended up using, but after some research this is 100% correct! We were afraid that too many syscalls during the query execution can produce huge performance hits so we decided to look for a different solution.

All in all, thanks for the comment!

I agree.

There is some basic knowledge on how to achieve the replication, but, in our case, a lot of it was improvised and we're not sure what is the best approach.

We tried to minimize the overhead (keep as little information as possible) and that was more or less the only guidance we used.

Well, consistency and availability are the problem here. While enforcing one of the property, you lose on the other. Not sure how would you achieve those opposing properties without introducing huge amount of complexity.

In this case, replication consists of communication between 2 instances, and when one them fails you need to pick consistency or the availability.

Paxos and Raft introduce consensus using multiple instances communicating with each other to mitigate those problems. Implementing correctly communication with 2 instances is hard enough, implementing it correctly for n instances is for sure harder.

In Memgraph, we actually tried to implement Raft and we did manage to some degree, but the effort was too much, especially when you look at the results we got (protocol that didn't work for many edge cases).