toniti

BuildKit(Docker build engine) maintainer here. It’s definitely our goal to limit the friction, so you get the benefits of containers but with similar performance as running things on the host. For the points you listed: BuildKit sends context incrementally between builds that transfers only deltas and not the whole context. Gzip is only needed when you want to push the image to a registry that doesn’t happen as often as your dev builds. For the hashes, we do need to calculate some for build cache (that of course makes your builds faster than they would be without it). For some bigger objects like layers we can defer checksum computation to push phase similarly to compression.

Another thing we have done is added cache mounts that let you keep application-specific cache between builds. This is usually, where the speed benefit from running things on the host used to come if your tools wrote some cache under $HOME etc.

There is still a constant container sandbox initialization time ~100ms, with some room to optimize that as well. If you have good examples of builds that can’t be easily optimized in containers would love to get your feedback/examples on the issue tracker.

Using `COPY --chmod` is not the correct solution for this. It works, of course, but it isn't very logical from Dockerfile readability standpoint. The real issue is the incorrect use of multi-stage builds. In multi-stage builds you define additional build stages where you prepare your binaries(eg. compiling them) and copy to the final runtime stage, so your final stage remains clean of temporary files created by your build steps. Based on your comment in your current build stage you run curl, extract etc., but you don't actually finish preparing the binary by correcting the executable bit. Instead, you copy the half-prepared binary to runtime stage and then try to continue your further modifications there. Eg. similarily if you would skip extracting step, copy the zip instead and extract it in runtime stage and then you would have the zip and the final binary in your exported image.

Another red flag is that you run `apt-get` after copying the binary to runtime stage(because you still want to tweak the binary there). That means any time source for binary changes, the `apt` commands need to run again and are not cached. If you just add the executable bit in your build stage you can reorder them, so the `COPY` comes after `RUN`.

As I mentioned you need to have the images you use available locally (eg. `docker images | grep docker/dockerfile`). In your Dockerfile you explicitly say that you want to use docker/dockerfile:1.0-experimental from the hub as a build frontend. If there is no such name/tag locally it needs to check the state in the registry as `1.0-experimental` tag is updated on new releases.

I recommend you to upgrade to BuildKit. Also, the above isn't really true for Docker CLI, mtime was never taken into account in the old builder and Docker CLI never sends uname/gname. With API it was possible in the old builder though (hence the incompatibility with compose for example). In more practical cases I've seen it caused by git directory instead, that isn't stable. In that case, you can put .git into .dockerignore or if you don't use it, BuildKit will discard it automatically.

I assume you mean the helper for COPY/ADD is pulled from the registry. That is not the case since Buildkit v0.5 / Docker v19.03 . If you have images you use locally, network connectivity is not required, otherwise, BuildKit will verify that the mutable tags have not changed in the registry (eg. it doesn't use the string "latest" to verify the validity of cache).

> Solution: you need to tag and push the build-stage images too

With BuildKit cache for all the intermediate stages is tracked. You can push the whole cache with buildx or inline the intermediate cache metadata in buildx or v19.03. `--cache-from` will pull in matched layers automatically on build. Can also export the cache to a shared volume if that suites you better than a registry.