throwaway20371
Joined 5 years ago
Joined 5 years ago
Actually, now that I think of it, that's not the problem. The problem is we keep changing the software. My laptop from 15 years ago still functions exactly the same way it used to. It hasn't disintegrated into a puddle of bits. You just can't use it to visit any "modern website" or run any "modern software". If we just stopped upgrading everything every 5 seconds we could keep using old technology.
If you don't want to repeat the steps from the beginning, you could make a completely separate checklist to be followed on a given system that includes things like "make sure X package is installed", "make sure Y configuration is applied", so that the new checklist accounts for any inconsistencies. This is pretty common anyway as checklists are broken up into discrete purposes and mixed and matched.
This brings up a very important point about checklists that I don't think gets enough attention.
The problem happens when somebody "updates" that web server in-place. If they try to record what changes they made in the middle of the checklist, eventually when someone tries the whole checklist from the beginning, they'll find it's now broken; the steps aren't working as expected. This happens to me when I try to record changes in my VirtualBox configuration after I add a new system package or something; later I try to re-deploy my vbox, and it breaks.
So checklists should be considered immutable. Once you create them, don't assume they will work again if modified. Instead, if you make any change to the checklist, you must follow all the steps from beginning to end. This way you catch the unexpected problems and confirm the checklist still works for the next person.
Soldiers also prefer trustworthiness over skill competency. Of course you want your brother-in-arms to do their job well, but it's more important that you can trust them with your life.
Technical skills are needed to work with a machine. People skills are needed to work with people.
* Documentation of the entire procedure is contained in one place. No need to go sifting through 20 different sources of documentation. This lowers the human emotional barrier to "just get it done", as people will always avoid things they aren't comfortable/familiar with, or don't have all the steps to. This central point of documentation also enables rapidly improving the process by letting people see all the steps in one place, which makes it easier to fix/collapse/remove steps.
* Automation in small pieces over time avoids the trap of "a project" where one or more engineers have to be dedicated to this one task for a long period of time. Most things shouldn't be automated unless there is demonstrably greater value in the cost of automating them than the cost of not doing so. Automating only the most valuable/costly pieces first gives immediate gains without sinking too much into the entire thing.
* One unified "method" to encapsulate any kind of process means your organization can ramp up on processes easier, reducing overall organizational cost.
* In the absence of any other similar process, you are guaranteed to save time and money.
I would say that the only potential downside is if someone decides to "engineer" this method, making it more and more and more complicated, until it loses its value. KISS is a requirement for it to be sustainable.
Cars used to be built by hand, had tons of bugs, and were expensive. Then a man came along and found a way to produce them faster, cheaper, and with less bugs. That was pretty amazing for a time, but they still had plenty of bugs. And then some people from a culture of very fastidious craftsmen obsessed with quality began producing cars a little cheaper, and with far fewer bugs, and they lasted much longer. Then the whole world realized, "shit, our machines don't actually need to be so fragile," and they followed suit.
The lessons learned by those people in that culture were promoted around the world, and evolved to shape what we now call Lean and Agile. But the people using these new processes forgot the first lesson: we don't have to accept the status quo.
One of the reasons for that constant maintenance seems to be The Web. Remember when you didn't need 4 gigs of ram to browse the web? When you didn't need a high-power 3D graphics card to look at Google Maps? (bad example but WebGL is mandatory for some simple sites, and if your graphics sucks/doesn't do hardware acceleration...)
I don't remember ever having to upgrade my car every few years just to visit a new local business. At some point we need to admit that this constant tech churn isn't improving our lives, but it is enriching some billionaires.
You don't have to support MongoDB, but you can support apps that were only written with Mongo as backend? That's awesome. I can't imagine it's production-ready yet but it's a great idea.
Clearly the problem isn't just having the skill or permission to change something, it's also the friction involved in figuring out how the hell to do it. How do you lower friction? Documenting things, making it easy to find things, making it easy to get access to things. If you can come up with an internal system that combines all of that, you have a one-stop shop for fixing high-friction problems.
I think Wikis are highly underrated. They seem to encapsulate all those things. Anyone can edit (or revert edits), anyone can access it, anyone can find it (eventually). Somehow we need to tie all the rest of an organization into a Wiki.