I think people kind of boiled themselves alive with Docker and don't step back to think if they're where they want to be often enough.
Docker first started getting traction when people were building their software with make. Make never quite got caching right (not really its fault), so nobody was really sure that their changes were going to show up in their release unless they ran "make clean" first. And, you had to have 700 helper utilities installed on your workstation for a build to work after "make clean". Docker to the rescue! It gives you a fresh Linux distribution, apt-get installs all the utilities needed for your build, and then builds your thing. It works 100% of the time! Celebrate!
At the same time, programming languages started moving towards being build systems. They want "mylang run foo.ml" to be fast, so they implemented their own caching. But this time they did it right; the "mylang" compiler knows about the effect every input file has on every output file, so it's guaranteed to give you the right answer with or without the cache. Some languages are so confident these days that you can't even disable the cache! They know it works perfectly every time. The result is extremely fast incremental builds that are just as reliable as clean builds, if you have access to that cache.
This, unfortunately, is not something that Docker supports -- layers have one input filesystem and one output filesystem, but now languages are producing two outputs -- one binary for the next layer, one cache directory that should be used as input (opportunistically) to the next build. The result is, to work around people writing makefiles like "foo.o: foo.c" when they actually meant "foo.o: foo.c foo.h helper.h /usr/include/third-party-library.h /usr/lib/libthird-party.so.42", EVERYONE suffers. "mylang run foo.mt" takes a few milliseconds on your workstation, but 5 minutes in Docker. Your critical fix is hamstrung by your CI infrastructure.
There are a number of solutions to this problem. You could have language-specific builders that integrate with your CI system, take the source code and a recent cache as inputs, and produce a Docker container as output. (Systems like Bazel even have a protocol for sharing cache between machines, so you don't have to copy gigabytes of data around that you probably don't need.)
But instead of doing that, people are writing articles about how their CI takes 3 hours when a build on their workstation takes 3 seconds, and it's because containers suck! But they don't actually suck -- the underlying problem is not saying "in production, we will only run .tar.gz files that contain everything the application needs". That is actually wonderful. The underlying problem is "the first build step is 'rm -rf /' and the second is 'make world'".
Docker first started getting traction when people were building their software with make. Make never quite got caching right (not really its fault), so nobody was really sure that their changes were going to show up in their release unless they ran "make clean" first. And, you had to have 700 helper utilities installed on your workstation for a build to work after "make clean". Docker to the rescue! It gives you a fresh Linux distribution, apt-get installs all the utilities needed for your build, and then builds your thing. It works 100% of the time! Celebrate!
At the same time, programming languages started moving towards being build systems. They want "mylang run foo.ml" to be fast, so they implemented their own caching. But this time they did it right; the "mylang" compiler knows about the effect every input file has on every output file, so it's guaranteed to give you the right answer with or without the cache. Some languages are so confident these days that you can't even disable the cache! They know it works perfectly every time. The result is extremely fast incremental builds that are just as reliable as clean builds, if you have access to that cache.
This, unfortunately, is not something that Docker supports -- layers have one input filesystem and one output filesystem, but now languages are producing two outputs -- one binary for the next layer, one cache directory that should be used as input (opportunistically) to the next build. The result is, to work around people writing makefiles like "foo.o: foo.c" when they actually meant "foo.o: foo.c foo.h helper.h /usr/include/third-party-library.h /usr/lib/libthird-party.so.42", EVERYONE suffers. "mylang run foo.mt" takes a few milliseconds on your workstation, but 5 minutes in Docker. Your critical fix is hamstrung by your CI infrastructure.
There are a number of solutions to this problem. You could have language-specific builders that integrate with your CI system, take the source code and a recent cache as inputs, and produce a Docker container as output. (Systems like Bazel even have a protocol for sharing cache between machines, so you don't have to copy gigabytes of data around that you probably don't need.)
But instead of doing that, people are writing articles about how their CI takes 3 hours when a build on their workstation takes 3 seconds, and it's because containers suck! But they don't actually suck -- the underlying problem is not saying "in production, we will only run .tar.gz files that contain everything the application needs". That is actually wonderful. The underlying problem is "the first build step is 'rm -rf /' and the second is 'make world'".