Backend: Allow Docker Layer Caching for Merged Results Pipelines

Problem to solve

Docker Layer Caching is a feature that allows for the reuse of previously built Docker image layers, resulting in faster image builds, reduced network bandwidth usage, and improved development productivity.

We support Docker Layer Caching out-of-the-box in our Auto-Build template for Auto-Devops. We use the --cache-from argument when building the image in the current pipeline to use the image built in the previous pipeline (via the $CI_COMMIT_BEFORE_SHA variable) and the image tagged as latest in the container registry as cache sources.

This works fine on regular branch pipelines but not on Merged Result pipelines as the pipeline runs in the context of the Merge Request itself and against an internal commit created by GitLab (from the docs):

GitLab creates an internal commit with the merged results, so the pipeline can run against it. This commit does not exist in either branch, but you can view it in the pipeline details.

Due to this, the $CI_COMMIT_BEFORE_SHA variable does not work as it returns that internal commit's SHA, and there is no image tagged with that SHA (yet).

Without being able to access the previous Merged Result pipeline's commit SHA we cannot programmatically designate it as a cache source, and thus the image built by the Merged Result pipeline is done from scratch every time resulting in long build times, inefficient CI minutes consumption, etc.

Proposal

Expose a way to reference a previous Merged Result Pipeline for a given source branch in the project, perhaps via a CI variable.

Once we store this variable then the AutoBuild job can just somehow be modified to use that reference as a cache source, the same way we currently do it for regular branch pipelines. This way we can easily use the commit SHA to use any image built by it as a cache-source.

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.