Here's a devil-in-the-details question that you might consider adding to your excellent article:
You have a web server in there, and also a storage system. What happens when the errors propagate up and the storage system dies? Does it force the entire node to reboot? Shouldn't the web server stay up to keep users informed that there is a serious problem, rather than simply going away? What's the best way to accomplish that?
You have a web server in there, and also a storage system. What happens when the errors propagate up and the storage system dies? Does it force the entire node to reboot? Shouldn't the web server stay up to keep users informed that there is a serious problem, rather than simply going away? What's the best way to accomplish that?