Comment by pelatimtt

Agree. And thing I noticed is that tools like #apache spark have become the de-facto standard for any data engineer work even when data size does not require it. Result is that many jobs are much harder to mantain and often slower (due to all the shuffling) than running on a single node.