This is the problem I have with almost all container orchestration systems (DC/OS, K8s, etc.) None of them can go from 1 node running all your apps to 100, in an easy straight forward manner. I wrote about this not too long ago:
https://penguindreams.org/blog/my-love-hate-relationship-wit...
I've also worked at both Kubernetes shops and DCOS/marathon shops and I honestly don't understand why people are choosing K8s. I'm sure a lot of it is the marketing and the power of Google behind it, but our DC/OS cluster had hundreds of nodes and could scale applications really well. Plus I found the marathon json files way less confusing. (Plus if you really want, you can use k8s to schedule containers on DC/OS, alongside marathon).
That being said, that DCOS platform team was 11 people and they had tons of custom scripts, wiki pages, labels and specific networking ingress/egress points that all worked together. (You could even use a label to specify if you wanted to run containers on the local VMWare cluster or in AWS; with nodes that auto-scaled out into AWS in high load).
I've met people at smaller startups who've gone the Nomad route instead, which seems a lot more sane in many ways, but still requires pretty careful planning and setup for large deploys.
https://penguindreams.org/blog/my-love-hate-relationship-wit...
I've also worked at both Kubernetes shops and DCOS/marathon shops and I honestly don't understand why people are choosing K8s. I'm sure a lot of it is the marketing and the power of Google behind it, but our DC/OS cluster had hundreds of nodes and could scale applications really well. Plus I found the marathon json files way less confusing. (Plus if you really want, you can use k8s to schedule containers on DC/OS, alongside marathon).
That being said, that DCOS platform team was 11 people and they had tons of custom scripts, wiki pages, labels and specific networking ingress/egress points that all worked together. (You could even use a label to specify if you wanted to run containers on the local VMWare cluster or in AWS; with nodes that auto-scaled out into AWS in high load).
I've met people at smaller startups who've gone the Nomad route instead, which seems a lot more sane in many ways, but still requires pretty careful planning and setup for large deploys.