zlib dirties the cache often
While checking at builds, I see thatzlib
dirties the cache, thus causing everything after that to be rebuilt. It has been happening noticeably frequently.
Investigation
Full investigation notes at #3988 (comment 638548232) (and its replies) and #3988 (comment 640594411) (and its replies). The TLDR is
-
omnibus
uses git tags to point towards cached state of the build directory. During each component build, it checks if the computed tag is present in the directory. If it is, we can skip building and just checkout that tag. - When computing the tag to use, the project name (
gitlab-ce
/gitlab-ee
) is a factor. - Another factor is the list of components built before the current component.
Even if we remove the project name component from tag computation, we can't guarantee the same component will be built at the same position during CE and EE builds (omnibus-ctl
for example). So, a CE and EE pipelne has the potential to invalidate cache of each other.
It is also difficult to keep track of the order of components being built, and maintain a balance whenever we add a new component. Hence, the easiest solution is for CE and EE to use different cache, thus not touching each other.