0x74696d

(author here) I'm one of the maintainers of HashiCorp's Nomad, so that example was likely inspired by the separation of duties that's part of our security model. In that environment, there's a subset of task (ex. container) configuration that's controlled by the cluster admin and a subset that's controlled by the job author deploying onto the cluster.

Author here! The motivating example of this post is frankly pretty lousy in retrospect (and was even so soon after writing, given the friendly reminder from Giovanni Campagna that `socket` wasn't one of the io_uring opcodes). At best this is an interesting limitation of seccomp. Maybe relevant if you were using gVisor?

This architecture is roughly how HashiCorp's Nomad, Consul, and Vault are built (I'm one of the maintainers of Nomad). While it's definitely a "weird" architecture, the developer experience is really nice once you get the hang of it.

The in-memory state can be whatever you want, which means you can build up your own application-specific indexing and querying functions. You could just use sqlite with :memory: for the Raft FSM, but if you can build/find an in-memory transaction store (we use our own go-memdb), then reading from the state is just function calls. Protecting yourself from stale reads or write skew is trivial; every object you write has a Raft index so you can write APIs like "query a follower for object foo and wait till it's at least at index 123". It sweeps away a lot of "magic" that normally you'd shove into a RDBMS or other external store.

That being said, I'd be hesitant to pick this kind of architecture for a new startup outside of the "infrastructure" space... you are effectively building your own database here though. You need to pick (or write) good primitives for things like your inter-node RPC, on-disk persistence, in-memory transactional state store, etc. Upgrades are especially challenging, because the new code can try to write entities to the Raft log that nodes still on the previous version don't understand (or worse, misunderstand because the way they're handled has changed!). There's no free lunch.

GKE clusters run on VMs, not on bare metal.

I'm the lead developer for Joyent of ContainerPilot, which is the tool at the core of our Autopilot Pattern implementation examples. The lifecycle events you recognize in Distelli are definitely similar. And Chef's new tool Habitat has a supervisor that was independently developed but ended up having interesting parallels with ContainerPilot. So there's a universal idea lurking under there, which is why we called Autopilot a "Pattern" rather than a tool in itself.

But it's not clear to me from a casual glance at the the docs whether Distelli lives inside the container during those hooks? That's part of the distinction of the Autopilot Pattern is making the higher-level orchestration layer as thin as possible.

(As far as the root, some of it is derived from my experiences as a perhaps-foolishly-early adopter of Docker in prod at my previous gig at a streaming media startup. The rest is derived from both principals with which Joyent's own Triton infra is built and our experiences speaking with enterprise devs and ops teams.)

> It’s not just deployment, but provides facilities for sharing code and extracting telemetry from your running apps. We believe that operations should be coupled with development, that developer tools should also be operational tools. For this reason, we’re keen to ensure that telemetry — metrics, logging, analytics, distributed tracing — is built-in from the beginning.

I think this is the most important problem IOPipe is trying to solve. Portability between platforms is a good goal but achievable with some discipline about avoiding the proprietary bits of the platform. Making "serverless" software operable to the same (or better) degree than traditional software deployment is the key.

Sending a swarm of unguided relativistic projectiles across the universe seems profoundly rude.

Yeah, that plus gvp. But if you dig into both they're just shell scripts under the hood (nice ones, though!) and given that I've got a makefile or shell script to build the container, run tests, etc. then adding a third-party tool is just one tiny bit more overhead.

> Makefiles, build: docker build Ugh, more Linux-Only. Don't follow this please.

Meh. I get your sentiment here, but I build and deploy on Linux and I post about the stuff I work on. I don't post about the Windows stuff that I haven't worked on in years. And I assume a level of intelligence in my readers that they can translate whatever might be generally applicable into their own platform in the same sense that I don't grumble about Raymond Chen's `Old New Thing` not being directly applicable to my own work but still enjoy it.

> The claim is that by not also solving versioning, go get is an incomplete tool. Stating that it was never intended to solve versioning is kind of beside the point.

You've summed up the entire reasoning behind my post more succinctly than I could. Thanks.

> now I have to check what VCS feature it uses, wether it uses a shell script, make , or build tool Z to fetch dependencies

The core of my work is on the operations side rather than on development (although I do a good bit of that as well), so my perspective is that this initial overhead is a one-time cost for using a library, and that I can live with that.

> I think it's time go devs acknowledge the fact that there is an serious issue here, instead of resorting to the ususal "you don't need that in go".

Encountering this and a lot of "well this is the way Google does is so we should too" was a big part of what inspired this post.

Author here. Happy to answer any questions about our setup.

Nice. If I was starting this project today (it's pretty mature at this point) that'd definitely be something I'd look into.

That being said, when Kinesis previewed it had only Java bindings for KCL. Not sure if that's still the case, but that'd be a limiting factor for our shop unfortunately.