sclevine
Joined 12 years ago
Karma
97
Submissions
23
Member since
Nov 3
[ my public key: https://keybase.io/sclevine; my proof: https://keybase.io/sclevine/sigs/sErO9lIpOe8rNyUDwHHfSOzH9uEJ4xdDFr22VVyGZ5Y ]
View on Hacker NewsRecent submissions
Comment · 5 years ago
This is especially true of Paketo's buildpacks, which tend to do exactly one thing each. E.g., Paketo Node.js buildpack is just a configuration file that's composed of other buildpacks: https://github.com/paketo-buildpacks/nodejs/blob/main/buildp...
Comment · 5 years ago
Additionally, Paketo's buildpacks build reproducible images given the same source code and buildpack versions.
Comment · 5 years ago
Don't hesitate to reach out on Slack if you have more questions: https://slack.buildpacks.io
A few tips on rebase:
(1) If you want to rebase without pulling the images first (so there's no appreciable data transfer in either direction), you currently have to pass `--publish`.
(2) If you need to rebase against your own copy of the runtime base image (e.g., because you relocated the upstream copy to your own registry), you can pass `--run-image <ref>`.
Comment · 5 years ago
Many comments here point out how difficult it is to manage a separate dependency stack for each container when you use Dockerfiles to build them. This problem is just as difficult, time-intensive, and security-critical for microservice apps running on K8s as it is for CLI tools and graphical apps.
Worth pointing out that there is an incubating CNCF project that tries to solve this problem by forgoing Dockerfiles entirely: Cloud Native Buildpacks (https://buildpacks.io)
CNB defines safe seams between OCI image layers so that can be replaced out of order, directly on any Docker registry (only JSON requests), and en-mass. This means you can, e.g., instantly update all of your OS packages for your 1000+ containers without running any builds, as long as you use an LTS distribution with strong ABI promises (e.g., Ubuntu 20.04). Most major cloud vendors have quietly adopted it, especially for function builds: https://github.com/buildpacks/community/blob/main/ADOPTERS.m...
You might recognize "buildpacks" from Heroku, and in fact the project was started several years ago in the CNCF by the folks who maintained the Heroku and Cloud Foundry buildpacks in the pre-Dockerfile era.
[Disclaimer: I'm one of the founders of the project, on the VMware (formerly Cloud Foundry) side.]
Comment · 6 years ago
I'm just using K8s (specifically: K3s) for configuration management in this case. This post hits the nail on the head: https://news.ycombinator.com/item?id=23006114
That said, NGINX can do UDP load balancing and WireGuard is stateless, so it should be possible to use this with a Service + NGINX ingress controller at scale: https://kubernetes.github.io/ingress-nginx/user-guide/exposi...
I have not tried it though.
Comment · 6 years ago
This was indeed the motivation for my write-up :)
Comment · 6 years ago
The OCI image format is a standardization of the Docker v2 image format, but they are generally compatible and interchangeable. That FAQ entry is misleading and slipped past the engineering team working on the project. Just removed it :)
Comment · 7 years ago
The presentation to the CNCF TOC covers some of the technical details: https://www.youtube.com/watch?v=uDLa5cc-B0E&feature=youtu.be
Some key points:
- CNBs can manipulate images directly on Docker registries without re-downloading layers from previous builds. The CNB tooling does this by remotely re-writing image manifests and re-uploading only layers that need to change (regardless of their order).
- CNB doesn't require a Docker daemon or `docker build` if it runs on a container platform like k8s or k8s+knative. The local-workstation CLI (pack) just uses Docker because it needs local Linux containers on macos/windows.
Comment · 7 years ago
While I really appreciate the work tonistiigi did to create the Cloud Foundry buildpack frontend for buildkit, it uses a compatibility layer[1] (which I wrote myself and no longer maintain) that only works with deprecated Cloud Foundry buildpacks that depend on Ubuntu Trusty. It doesn't work with the new, modular Cloud Native Buildpacks, and the buildpacks that ship with it are outdated (and vulnerable to various CVEs). It will stop working with new buildpack versions entirely when Cloud Foundry drops support for Trusty.
Implementing CNBs as a buildkit frontend would break key security and performance features. For instance, CNBs can build images in unprivileged containers without any extra capabilities, which buildkit cannot do. CNBs can also patch images by manipulating their manifests directly on a remote Docker registry. This means that image rebuilds in a fresh VM or container can reuse layers from a previous build without downloading them (just metadata about them), and base images can be patched for many images simultaneously with near-zero data transfer (as long as a copy of the new base image is available on the registry). As far as I know, buildkit can't do any of that yet.
That said, we do plan on using buildkit (once it ships with Docker by default) to optimize the CNB pack CLI when you build images without publishing them to a Docker registry. It's a huge improvement over the current Docker daemon implementation for sure!