Comment

Comment by thu2111

I dunno, because of writeups like GitLabs? We're lucky GitLab is so transparent. It lets us see that their claims don't appear to always match the actual reality.

Here's an example.

Blog post says: "After transitioning each service, we enjoyed many benefits of using Kubernetes in production, including much faster and safer deploys of the application, scaling, and more efficient resource allocation"

Wrong. Actual analysis by the engineers of one of their migrated jobs (urgent-other sidekiq shard) says their cost went from $290/month when running in VMs to $700/month when running in Kubernetes. They tried to use auto-scaling but failed completely and ended up disabling it:

https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/920...

Kubernetes looks like a massive LOSS in the case of this service. The perceived scaling benefits didn't materialise at all, and their costs more than doubled. They also spent a lot of engineering time optimising startup time of this job: 7 PRs and complex writeups/testing were required. Just so they could try to auto-scale a job to reduce the hit of the hugely multiplied base costs. If not for Kubernetes that eng time might have been spent adding features to the product instead.

It solves really problems out out the box like load balancing, ingress, cert management, autoscaling, health checks, autorepair

Most of these problems are created by Kubernetes, like "ingress" which is a term only Kubernetes and low level networking uses, "cert management" which often isn't required in more traditional setups beyond provisioning web servers with the SSL files, "health checks" are a feature of any load balancer and the whole point of VMs is to avoid the need to repair hardware. Finally the demand auto-scaling as seen in this case is (a) possible without Kubernetes and (b) not actually working well anyway.

Frankly this set of bugs, writeups, PRs etc is quite scary. I worked with Borg at Google and saw how it was both a force multiplier but also a huge timesink in many ways. It made some complex things simple but also made some simple things absurdly complex. At one point I had a task to just serve a static website on Borg, it turned into a three month nightmare because the infrastructure was so fragile and Google-specific. A bunch of Linux VMs running Apache would have been far faster to set up and run.

parent thu2111 6 years ago

Discussion

Live from the official Hacker News API

A modern reader for Hacker News · SvelteKit + Skeleton