sougou
Joined 10 years ago
Karma
44
Submissions
38
Member since
Aug 16
Recent submissions
Comment · 1 year ago
Hi HN, this is a series on a way to generalize consensus protocols and how to adapt them to existing systems.
This is part 7: Discovery and Propagation. With this, we complete everything about the protocol part of Generalized Consensus. There are a few more advanced topics to cover: They are needed for running in production.
One of our goals is to make this approach work for Postgres via Multigres. But the principles can be used to implement your own solutions also. This will make Durability and High Availability more accessible for users.
Comment · 1 year ago
Hi HN, this is a series on a way to generalize consensus protocols and how to adapt them to existing systems.
This is part 6: It covers Revocation and Candidacy, the prerequisites for a leader change. We explain how to revoke previous leaderships and recruit for a new candidacy. It's a bit complex, but there are plenty of animations to help you along.
One of our goals is to make this approach work for Postgres via Multigres. But the principles can be used to implement your own solutions also. This will make Durability and High Availability more accessible for users.
Comment · 1 year ago
Hi HN, this is a series on a way to generalize consensus protocols and how to adapt them to existing systems.
This is part 5: This one covers how to safely handle multiple coordinators racing to take action. We explore term numbers, coordinators as separate agents, and lock-free approaches.
One of our goals is to make this work for Postgres via Multigres. But the principles can be used to implement your own solutions also. This will make Durability and High Availability more accessible for users.
Comment · 1 year ago
Hi HN, this is a series on a way to generalize consensus protocols and how to adapt them to existing systems.
This is part 4: This one has a nice animation that shows how requests get fulfilled.
One of our goals is to make this work for Postgres via Multigres. But the principles can be used to implement your own solutions also. This will make Durability and High Availability more accessible for users.
I'll post here as I release the subsequent parts.
Comment · 1 year ago
Hi HN, this is a series on a way to generalize consensus protocols and how to adapt them to existing systems.
This is part 3: Governing Rules. This is the foundation for the rest of the series. We'll be repetitively applying these rules to map out all parts of the protocol. Also, every system that I know of implicitly follows these rules.
One of our goals is to make this work for Postgres via Multigres. But the principles can be used to implement your own solutions also. This will make Durability and High Availability more accessible for users.
I'll post here as I release the subsequent parts.
Comment · 1 year ago
Hi HN, this is a series on a way to generalize consensus protocols and how to adapt them to existing systems. I've released two parts so far. They explain the problem and the requirements. You can follow the link to the second part from the first part.
One of our goals is to make this work for Postgres via Multigres. But the principles can be used to implement your own solutions also. This will make Durability and High Availability more accessible for users.
I'll post here as I release the subsequent parts.
Comment · 1 year ago
I oversaw this work, and I'm open to feedback on how things can be improved. There are some factors that make this particular situation different:
This was an LLM assisted translation of the C parser from Postgres, not something from the ground up.
For work of this magnitude, you cannot review line by line. The only thing we could do was to establish a process to ensure correctness.
We did control the process carefully. It was a daily toil. This is why it took two months.
We've ported most of the tests from Postgres. Enough to be confident that it works correctly.
Also, we are in the early stages for Multigres. We intend to do more bulk copies and bulk translations like this from other projects, especially Vitess. We'll incorporate any possible improvements here.
The author is working on a blog post explaining the entire process and its pitfalls. Please be on the lookout.
I was personally amazed at how much we could achieve using LLM. Of course, this wouldn't have been possible without a certain level of skill. This person exceeds all expectations listed here: https://github.com/multigres/multigres/discussions/78.
Comment · 1 year ago
We absolutely need this!!! I vote yes :).
sugu [at] supabase [dot] io