botswana99

Well, I find this post looks good, but a like lot of 'data for developers' posts it's just a list of tools. As if a collection of tools banded together actually makes your customer successful. What's missing? 1. There's nothing about deployment. How do I take this collection of tools and code and actually deploy it into production, or actually regression test it functionally? How do I make a small change in a database table and not have a massive regression? How do you do that automatically? How do you do it quickly? 2. It's cursory on testing. One of the biggest differences from a software developer to a data engineer is that your data providers give you crap data all the time. It could break. How do you test data? How do you get adequate test coverage? These things are essential for software developers and are actually doubly essential for data engineers and building analytics systems. 3. It's what success looks like. It's not just about a collection of tech; it's about making your customers successful. What does it mean to deliver good insight? How do you do it? How do you measure customer success, and measure your success? As a team, you wouldn't talk about software engineering without mentioning DevOps or DORA metrics. There's nothing here about that.

sigh

I hope so, plumbers are damn expensive

The SAT, like general intelligence, is half hard work and half inherited ability, from what I've read. Characteristics that you have no control over, like height, intelligence, race, or sex, should not determine your wealth. It should really be about hard work and character; that's fair. At least in the U.S., there are too many industries like law, venture capital, or VC-invested startup founders, where the pedigree of your school is what matters to your success.

Neither statement is true, unfortunately. The reports that you read in the media don't reflect the state of the world. In the US, most plumbers and SWE make between 100-200K. The way to wealth is to start one's own business. Hence, hard work and character.

The dude has obviously never met a multimillionaire owner of an auto body shop, a house demolition business, or a plumber. Did they do well in high school calc class? No. Are they richer than the author? Yes

Blowing up the cognitive hierarchy is a gift that AI gives us. Let's move into an age where hard work and character matter more than your SAT score at 17.

Many data teams often find themselves as 'tool jockeys' instead of becoming true engineers. They primarily learn some company data, and then rely on drag-and-drop or YML configuration functionality within the constraints of the tool's environment.

Their organization often insists they must use standard tools, and their idea of a good job is that the task works fine within their personal version. No automatic testing, no automated deployment, no version control, and handcrafted environments. And then they get yelled at when things break and yelled at for taking too long. And most DEs want to quit the field after a few years.

The real question is not that DE and software engineering are converging. It's why most DEs don't have the self-respect and confidence to engineer systems so that their lives don't suck.

My mantras in management:

Love Your Errors

It's not a failure, it's an opportunity for improvement

No shame, no blame

You should be proud…you’ve learn the difference between the pursuit of value vs. the pursuit of valuation. Congrats.

Jesus H Christ, the Universe is big.

So, this explains the average aging Trump voter's cognitive impairment? Well, at least it's not leaded gasoline or reality TV.

This is great. I have personally gone through a half a dozen toasters. It such a waste.

I want to buy one for my lifetime