It always amazes me how readily people assume that their problem is unique and special. This article can be summarized as: standard software engineering problem gets solved with standard software engineering practices. The problem they describe of a giant function that gets touched every time something changes is called a "God Function" or a "God Class" and it's one of the most common problems in software engineering. Their solution is called "Dependency Inversion" and it was designed to solve literally that exact problem in the exact way they're doing it.
That doesn't mean it's easy! Anyone who applies Software Engineering principles to make their code better deserves congratulations. So congratulations, Stitchfix! But you could have found this solution a lot quicker if you didn't fall into that age old trap of thinking your data-science code was different, and excluded from standard software engineering. Everyone does this, and I don't know why. They read all these blogs and books on software engineering best practices, but then they don't apply them to anything. You can't apply it to your SQL because it's just different. Your database organization is different. Your I/O code is different. Your API surface is different. Your UI is different. Your feature engineering is different.
No! It's software. Stop thinking that you need a unique solution to your unique snowflake problem. Apply well-known and good software engineering practices to everything you ever do.
They even say in the article "It hasn’t grown convoluted out of ... bad software engineering practices". Of course it has! What do you think software engineering practices are supposed to apply to!?
That doesn't mean it's easy! Anyone who applies Software Engineering principles to make their code better deserves congratulations. So congratulations, Stitchfix! But you could have found this solution a lot quicker if you didn't fall into that age old trap of thinking your data-science code was different, and excluded from standard software engineering. Everyone does this, and I don't know why. They read all these blogs and books on software engineering best practices, but then they don't apply them to anything. You can't apply it to your SQL because it's just different. Your database organization is different. Your I/O code is different. Your API surface is different. Your UI is different. Your feature engineering is different.
No! It's software. Stop thinking that you need a unique solution to your unique snowflake problem. Apply well-known and good software engineering practices to everything you ever do.
They even say in the article "It hasn’t grown convoluted out of ... bad software engineering practices". Of course it has! What do you think software engineering practices are supposed to apply to!?