Caching general build artifacts between stages

Problem to solve

This issue "Caching general build artifacts between stages" describes a fundamental problem in GitLab CI where users need an efficient way to share files between different stages of a pipeline.

The core problem is that GitLab CI, by default, starts each stage with a clean environment, which means files created in one stage (like compiled binaries or build outputs) are not automatically available in subsequent stages. This forces users to either:

1. Use artifacts - which uploads files to the GitLab server and then downloads them in the next stage (causing bandwidth usage, storage costs, and time delays)
2. Use cache - which was designed for dependencies like vendor folders, not for passing build outputs, and is not guaranteed to be available

Users are looking for a more efficient solution that would:

• Allow files to be passed directly between stages without uploading/downloading
• Not expose intermediate build files as downloadable artifacts in the UI
• Reduce build times by avoiding redundant work
• Potentially keep jobs running on the same runner to maintain state

This issue has been open for several years with many users expressing the need for a "sticky runner" feature or some form of local artifact passing that doesn't require transferring files to and from the GitLab server.

The most common workarounds include using artifacts with expiration, using cache with specific keys, or setting GIT_CLEAN_FLAGS to prevent cleanup between stages when using a single runner.

Original comment thread

I'm not sure why it's not working, maybe I did understand the use case wrong for cache.
What I wanted to do:
Define 2 stages => build and deploy
In the build stage our app gets put together and into a new folder ./build/
This folder needs to be carried over to the deploy stage so it can be deployed on a server.

Here's what the .gitlab.yml file looks like (shortened)

image: ctsmedia/gitlab-runner-build-base-cts:latest
stages:
  - build
  - deploy

before_script:
  - export DEBIAN_FRONTEND=noninteractive
...

build:
  stage: build
  artifacts:
    paths:
      - build/
  cache:
    paths:
      - build/
  script:
    - phing purge
    - sh bin/ci-merge-feature-branches.sh
    - phing build

dev:
  stage: deploy
  script:
    - apt-get -qqy install rsync
    - rsync -ae ssh ./build/* user@IP:/super/secret/path
  only:
      - /^issue-.*$/
      - /^iss#.*$/
      - /^feature.*$/
      - develop

The build run succeeds but the build folder gets deleted when switched to the deploy stage:

gitlab-ci-multi-runner 0.7.2 (998cf5d)
Using Docker executor with image ctsmedia/gitlab-runner-build-base-cts:latest ...
...
Running on runner-5dfeb163-project-2-concurrent-0 via 1112eb26ef3f...
Fetching changes...
Removing artifacts.upload.log
Removing build/
Removing node_modules/
HEAD is now at 2d7994e Merge remote-tracking branch 'refs/remotes/origin/issue-5' into develop
Checking out ceae231a as develop...

For now I just use one stage instead of which is perfectly fine and has less overhead because there is only run to be made. The build folder and all it's content is not tracked and gets created during the build step.