Add cache for remote includes

What does this MR do and why?

Adds a cache option for remote includes to allow caching fetched content for a specified duration, reducing redundant network requests and improving pipeline performance.

When enabled, remote CI configuration files can be cached with either:

  • A boolean cache: true (uses default 1 hour TTL)
  • A duration string like cache: '1 day' or cache: '30 minutes' (minimum 1 minute)

This feature helps reduce load on external servers hosting CI templates and speeds up pipeline creation when the same remote includes are used frequently.

Feature flag: ci_cache_remote_includes - this will be quite a quick rollout.

References

Closes https://gitlab.com/gitlab-org/gitlab/-/issues/351252

How to set up and validate locally

  1. Enable the feature flag: Feature.enable(:ci_cache_remote_includes)
  2. Create a .gitlab-ci.yml with a cache:
include:
  - remote: 'https://gitlab.com/gitlab-org/gitlab-foss/-/raw/master/.gitlab-ci.yml'
    cache: '1 day'
  1. Create multiple pipelines and observe that the remote file is only fetched once
Edited by Laura Montemayor

Merge request reports

Loading