This was a great read. I've been a sysadmin/SRE/Devops/Platform/infra ( Why are there so many different titles) engineers for many years, for the past ~5 years I've been using Kubernetes and have helped companies migrate lots of workloads to k8s. I've worked with many people who dislike k8s, but almost all of them had almost no experience with k8s. A lot of the anti-k8s people would eventually build out something similar to k8s, which made me love [this blog post](
https://www.macchaffee.com/blog/2024/you-have-built-a-kubern...) that helped explain that more.
Anyway, I love that you are/were skeptical of Kubernetes, but then looked into k8s more; we need more people like that in the world.
I think you did a great job explaining the internals of Kubernetes and then comparing it to "not" k8s. e.g. when you said the following:
> Compared to "classic" clusters, a Service provides a similar abstraction to the combination of:
** N nodes running Keepalived + NGINX (with a dynamic set of backends)
** CNAME records for the virtual (Keepalived) IP
It's a great point because a lot of anti-k8s people just say "Just put your service on a few VMs and put a LB in front of it". But if you don't just pay someone else to do it (ALB,NLB,etc) you gotta do something like Keepalived + NGINX like you mentioned, and then you are one step into creating your own k8s.
I also love how you still brought up issues with k8s. For as much love and experience I have of k8s, I feel like there is some odd behavior it has as defaults. e.g. the default is no cpu or mem requests (don't get me started on storage). I've worked many places where an engineers app that "shouldn't" use a ton of resources used up enough to cause the k8s worker node to have issues, which then caused other pods to fail.
Anyway, I love that you are/were skeptical of Kubernetes, but then looked into k8s more; we need more people like that in the world.
I think you did a great job explaining the internals of Kubernetes and then comparing it to "not" k8s. e.g. when you said the following:
> Compared to "classic" clusters, a Service provides a similar abstraction to the combination of:
It's a great point because a lot of anti-k8s people just say "Just put your service on a few VMs and put a LB in front of it". But if you don't just pay someone else to do it (ALB,NLB,etc) you gotta do something like Keepalived + NGINX like you mentioned, and then you are one step into creating your own k8s.I also love how you still brought up issues with k8s. For as much love and experience I have of k8s, I feel like there is some odd behavior it has as defaults. e.g. the default is no cpu or mem requests (don't get me started on storage). I've worked many places where an engineers app that "shouldn't" use a ton of resources used up enough to cause the k8s worker node to have issues, which then caused other pods to fail.