We recently ran a large clustering job over billions of records (and a few TBs of data on spinning disks) on a single machine with minimal command line tooling in a few hours. Not really optimized yet. People forget how fast modern hardware is and overestimate how much useful data they have (or need).
I think I should start a company around minimalistic data tooling or the like - the amount of waste seems large across the industry.
I saw the de-skilling a few year ago, where a guy stiched together a compete application from a couple of SaaS APIs. Cool, but it somehow does not impress me.
I think I should start a company around minimalistic data tooling or the like - the amount of waste seems large across the industry.
I saw the de-skilling a few year ago, where a guy stiched together a compete application from a couple of SaaS APIs. Cool, but it somehow does not impress me.