Comment by DanielHB

Man I tried this approach by making my builds dockerized, turns out docker layer caching is pretty slow on CI and adds a lot of overhead locally.

Do not recommend this approach (of using docker for building).