CI: Caching outside repository seems impossible

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Initial problem description:

Apparently I cannot upload any cache of, say, /usr/local/lib/python3.5/site-packages, where the packages are stored when installing dependencies on the python:3 container.

Core Problem (GitLab Duo summary)

Users cannot cache files stored outside the project directory (e.g., /usr/local/lib/python3.5/site-packages), receiving the error "not supported: outside build directory."

Specific Use Cases Affected

Package Manager Dependencies:

  • Python: Cannot cache pip packages in /usr/local/lib/python3.5/site-packages
  • Maven/Gradle: Cannot cache dependencies in ~/.gradle/ and ~/.mvn/
  • Composer (PHP): Cannot cache in ~/.composer/cache/
  • npm/Yarn: Cannot cache $HOME/.npm or npm cache directories
  • Go modules: Cannot cache $GOPATH/pkg/dep
  • Rust/Cargo: Cannot cache /usr/local/cargo/registry
  • Nix: Cannot cache /nix/store

Impact on Workflows:

  • Slow builds: Users must re-download dependencies for each build
  • Increased network traffic: Repeated downloads waste bandwidth
  • Integration testing issues: Composer integration tests fail when packages can't be properly structured
  • Docker image workflows: Can't cache artifacts when using pre-built images with GIT_STRATEGY: none

Current Workarounds (with limitations)

  • Symlinks: Don't work reliably (referenced issue #3385 (closed))
  • Environment variables: Redirecting cache paths (e.g., XDG_CACHE_HOME, GRADLE_USER_HOME) to project directory
  • Manual copying: Moving files in/out of cacheable directories in before/after scripts
  • Custom Docker images: Maintaining separate images with pre-installed dependencies

User Frustration Points

  • 10-year-old issue: Originally opened in 2015, still unresolved
  • Comparison to competitors: Other CI systems (GitHub Actions, Travis) support this functionality
  • Goes against best practices: npm and other tools recommend caching outside project directories
  • No clear timeline: Issue remains in backlog with no apparent progress

The fundamental limitation is that GitLab Runner's caching mechanism only supports paths within the build directory, forcing users into inefficient workarounds that increase build times and complexity.

Sources: CI: Caching outside repository seems impossible

Edited by 🤖 GitLab Bot 🤖