Runner job volumes

Joe Burnettrequested to merge
jburnett/runner-job-volumes into main

Why is this change being made?

This MR adds a design document blueprint for GitLab Runner Job Volume Mounts, a new feature that enables CI jobs to mount GitLab repositories and artifacts as filesystem volumes through .gitlab-ci.yml configuration.

Background

Current CI workflows requiring multi-repository access or artifact consumption face significant limitations:

  • Full Git Clones take 60-900 seconds for sparse access patterns, requiring download of entire repositories
  • Artifact Downloads require manual scripting with no filesystem integration
  • Multi-Repository Builds need complex workarounds using git submodules or manual cloning
  • Cross-Project Dependencies have no native way to access related repositories in CI jobs

Proposed Solution

This blueprint proposes leveraging GitLab FUSE with copy-on-write capabilities to provide instant, lightweight access to repository content and build artifacts across all GitLab Runner executors. The solution offers:

  • 15-300x faster repository access for sparse workflows
  • 90%+ reduction in bandwidth and storage usage
  • Native filesystem integration for repositories and artifacts
  • Seamless multi-repository development workflows
  • Cross-executor support (Docker, Shell, Instance, Docker+Machine, Docker-Autoscaler)

This design document establishes the architecture, security model, implementation strategy, and use cases for this capability.

Author and Reviewer Checklist

Please verify the check list and ensure to tick them off before the MR is merged.

  • Provided a concise title for this Merge Request (MR)
  • Added a description to this MR explaining the reasons for the proposed change, per say why, not just what
    • Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
  • Assign reviewers for this MR to the correct
    • The when to get approval handbook section explains when DRI approval is required
    • The who can approve handbook section explains how to identify the DRI
    • If the MR does not require DRI approval, consider asking someone on your team, such as your manager.
    • The approver may merge the MR. If they approve but don't merge, you can merge.
  • For transparency, share this MR with the audience that will be impacted.
    • Team: For changes that affect your direct team, share in your group Slack channel
    • Department: If the update affects your department, share the MR in your department Slack channel
    • Division: If the update affects your division, share the MR in your division Slack channel
    • Company: If the update affects all (or the majority of) GitLab team members, post an update in #whats-happening-at-gitlab linking to this MR

Related Issues and Documentation

Merge request reports