nrh

needs the ball sizes to represent the storage capacity!

I'd love to try this, but most of the places that I would want to use it are servers, and the rust requirements are way beyond where debian-stable lives.

Too much hassle to muck with backports and package pinning for a QoL tool - my feedback would be to try to make this install without tomfoolery on the stable branch of major distros.

I'm pretty into this place's business philosophy: just ship us your stuff with a little bit of information and we'll repair it. don't call ahead, don't ask for a quote, don't send us garbage that's not worth repairing. mucking around and/or unnecessary communication will just slow things down. We'll fix it and ship it back and bill you. End Of Story. I wish my work-life were so simple.

ATE0!

Once upon a time I worked for what was (once) the world's largest BBS. Remember the US Robotics Courier HST? 16.8kbps! Now imagine a room filled with shelf after shelf of Couriers, several hundred individual lines, at least a few hundred amps worth - all negotiating vigorously, clicking on-hook and off, all the time. Fun times.

Pretty sure I could still tell the difference between a 14.4k and 16.8k negotiation by ear.

I'd say the data platform overall, bigquery is certainly great . So are some other bits.

Spotifier here. For the record, I hate puppet with the fiery intensity of a thousand suns. It's also pretty hard to magically make go away. I could probably do a whole talk on why puppet is difficult to kill, it's the kudzu of config management. I think it is the worst.

We've got our warts and a pile of tech debt, and I wouldn't want anyone to think otherwise. Containers are a part of a long-term strategy to get away from puppet and onto more idempotent units of deployment, and move a lot of what is considered to be "configuration" back into the build process where it belongs.

Spotifier here. Frankly, price is not the biggest factor in a decision like this. If we were going for the lowest cost cloud option, it probably wouldn't be either AWS or Google - there are other providers who are hungrier for business that would be willing to do deep cuts at our scale.

The way we think about this is that there are basically two classes of cloud services: commodities and differentiated services. Commodities are storage/network/compute, and the big players are going to compete on price and quality on these for the foreseeable future (as with most commodities).

The differentiated services stuff is a bit more interesting. Different players have different strengths and weaknesses here - AWS has way, way better capabilities when it comes to administration and access control and identity management, for example (which is actually pretty important when trying to do this in a large org). The places were Google is strong (data platform) are the places that are most important for us as a business.

Compelling: dataproc+gcs, bigquery, pubsub, dataflow Made it safe: high-enough quality, cheap enough.

What more would you like to know?

Spotifier here. This is an important point. I have nothing bad to say about Cloudera or HWX (disclaimer: we're an HWX customer - we've had a pretty good experience), but I don't really see a compelling reason at this stage to manage your own cluster(s) (HIPAA/regulatory constraints, maybe?)

Getting shared-storage and indepedently operated/scaled compute clusters on top of that storage isn't easily achievable with the standard Hadoop stack, and building that on top of HDFS is non-trivial.

Spotifier here. One of the big risks that we identified with the Google partnership was exactly this: Google isn't exactly known for awesome customer service.

We've been very pleasantly surprised. The cloud team has been pretty awesome to work with, including lots of engineer<->engineer contact, walk-throughs of systems and code for critical dependencies, and solid support and collaboration. Exceeded our expectations.

  To fix this problem, we introduced the simple rule that new 
  stable releases of the software must (a) talk to old stable 
  releases and (b) they must support existing apps, without 
  changes. We more or less succeeded with that, so ZeroMQ 
  versions 3.2 and 4.0 work nicely with 2.2 and 2.1, for  
  example.

...and pain ensued.