It seems like long builds are either (a) necessary or (b) user error. (a) If you have a tree of dependencies and you change the root, you should rebuild everything that depends on it to make sure it's still compatible. (b) if you placed your application into one of the initial Dockerfile layers, but then you're installing dependencies that don't depend on you, it's user error.
What's the situation where your application needs to go first in the Dockerfile, and then you need to put a bunch of stuff that doesn't depend on your application?
What's the situation where your application needs to go first in the Dockerfile, and then you need to put a bunch of stuff that doesn't depend on your application?