marcoceppi

Since the GPU libraries are hosted on the node, privileged flag is typically required to make that possible. I'm sure there will be improvements to not require privileged, but today it's mostly a requirement to get anything useful out of containers tapping into GPU.

That said, if you set the allow-privileged flag to false GPU drivers will still be installed but you may not be able to make use of the cuda cores

Gluster is a good alternative, Ceph can be prickly if you don't want a block device and instead just need a filesystem. CephFS fills sort of the same role, but does so on top of Ceph rbd.

Since GlusterFS /can use/ NFSv4 as a client, it should work with the stuff @samco_23 uses

I'm curious what init system would you prefer?

OpenShift is an interesting product, because it's not just Kubernetes, but a PAAS on top of Kubernetes packaged as a single solution. As a result you're not using k8s, but OpenShift / Origin. This has pros and cons which mainly boild down to simplicity vs lock in.

Honestly, I'd suggest you shop around, comparing CDK to OpenShift would be kind of like comparing virtual machines to a cloud provider. In that, virtual machines are implemented in a ton of ways, but cloud providers are a platform which abstracts that whole VM layer and provides a product on top of VMs.

Much in the same way we're packaging up and distributing that underlying engine. There's a lot of flexibility with that. You can use Kuberentes directly, you can leverage Helm which is what a lot of the community seems to be moving towards for package management on k8s, and then things like Deis and others implement that PAAS layer similar to OpenShift.

Why I'd recommend this method over OpenShift is the flexibility we afford you. For starters, things like Helm and Deis don't come as a part of CDK but will work out of the box on it. Since it's just delivering Kubernetes the platform. As such, if you try Deis and don't like it, you could try Kel or any other PAAS / tool built to work against vanilla Kubernetes. With OpenShift - you use it and that's the choice you're saddled with.

At the end of the day, take some time to try both. Deis is quite polished and would give you an idea of what to expect from other PAAS solutions. It's built on Helm which is part of Kubernetes so it's less deltas from what upstream is doing.

We don't begin integration testing at day 0 of release, we are constantly integration testing.

That said, five days is a goal of ours, not a hard deadline or SLA. An example of this was the 1.5.2. This came out 8 or 9 days after upstream because we had failing integration and e2e tests starting a few days before release and it took some time to hunt down, address, and make sure it was an issue with how we're deploying k8s and not k8s itself.

Hopefully that clarifies how we're able to to stride towards that goal. To be honest, testing like this is as much and art as it is a science and it's taken quite a while to get to where we are now (and admittedly, still a ways to go until we're satisfied).

Let me know if have any other questions!

First off, sage math looks awesome \m/. This is a great question, I'll be making sure we update our docs and pages to highlight these differences, here's a few that might be helpful for you:

From an installation standpoint you get the same Kubernetes /everywhere/. You can run this on all the public clouds, e.g. AWS, GCE, Azure, Rackspace, etc., and on private infrastructure like OpenStack, VMWare, and bare metal. And any Ubuntu machine, like a laptop, single server, etc.

CDK as a whole is really just applying operational knowledge around Kubernetes. We are not adding any additional bits to k8s, we distribute the same binaries as upstream. If you stop using our tools, you can still manage the cluster as if you'd stood it up by hand. Since we're distilling operations, and not just installation, we cover the entire breadth of lifecycle tasks. Come 1.5.3, 1.6.0 (+ etcd3), and beyond we work to make sure tasks like upgrades work reliably and consistently.

We also present a consistent interface to common maintenance tasks, so that everyone using the solution uses the same primitives for maintenance. This helps eliminate the need to SSH and hunt around for these tasks. All the operational code we produce is open source, it lives in the upstream tree, and we love contributions!

Our roadmap for features is driven almost exclusively by our users, both community and commercial.

I could probably write pages, but I think those are probably the starting points for things we're doing as a distribution of Kubernetes. If you have any specific issues while running k8s, I'm happy to help answer how we handle those, if we do.