Comment

Comment by capplexham

The paradigm shift from ETL to ELT is what went wrong with Data Lakes. An enormous amount of business logic and effort was historically accumulated in the Extract-Transform stage (usually in COBOL or SQL), and then Loaded into a Data Mart for end-users.

When ELT became the corporate "Strategy", Extract-Load was technically easy and had a lot of Big Data evangelism and exuberance behind it. The money flowed easily at the beginning, but budgets aren't unlimited and corporate knowledge isn't easy to retain. So Data Lakes became Swamps of just Extract-Load, and business users were told to transform the data themselves.

As someone who lived through it for 12 years at an Insurance company, I like to think of it as ETL becoming Extract-Load-GoFuckYourself.

parent capplexham 3 months ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton