Comment

Comment by d_burfoot

The Apache big data suite (Hadoop/Spark/Yarn/Hive/HDFS/etc).

In several years of big data engineering work, I've believe I've seen only one application that couldn't be refactored into a simple multi-instance framework-free program. People use the big data frameworks as glorified distributed-job management tools, and the resulting systems are more fragile, more complex, more vulnerable to weird version compatibility errors, and less efficient.

parent d_burfoot 6 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton