Cache: rename local artifact when FF_HASH_CACHE_KEYS is toggled

What does this MR do?

cache: rename local artifact when FF_HASH_CACHE_KEYS is toggled

When FF_HASH_CACHE_KEYS is enabled, the local cache artifact is stored under a SHA256-hashed path. When the FF is disabled, it uses the sanitized (unhashed) path. Toggling the FF previously caused the existing local artifact to be ignored, forcing a full re-archive.

Introduce --alternate-file to cache-archiver: if the primary file does not exist but the alternate does, rename it before archiving. This enables upgrade (unhashed → hashed) when enabling the FF, and downgrade (hashed → unhashed) when disabling it, so existing local cache artifacts are reused across FF changes.

Why was this MR needed?

To support smooth transition when feature flag is toggled.

What's the best way to test this MR?

Test 1: Execute cache-archiver command

Execute cache-archiver command and confirm it renames the artifact file.

cd gitlab-runner
make runner-bin-host

Preparation:

$ mkdir -p /tmp/test-project/some-dir
$ echo "test content" > /tmp/test-project/some-dir/file.txt
$ cd /tmp/test-project && zip cache-hashed-abc123.zip some-dir/file.txt

Before

$ ls -l /tmp/test-project
total 8
-rw-r--r--@ 1 taka  wheel  197  3月 23 13:47 cache-hashed-abc123.zip
drwxr-xr-x@ 3 taka  wheel   96  3月 23 13:47 some-dir

Execute command:

  • This is FF=Off scenario, because it's passing the unhashed file as a --file and the hashed file as a --alternate-file.
  • It should rename the hashed file to unhashed one.
$ ./out/binaries/gitlab-runner-darwin-arm64 cache-archiver \
      --file /tmp/test-project/cache-unhashed.zip \
      --alternate-file /tmp/test-project/cache-hashed-abc123.zip \
      --path /tmp/test-project/some-dir

Runtime platform                                    arch=arm64 os=darwin pid=50447 revision=8eac319c version=18.10.0~pre.791.g8eac319c
Renamed /tmp/test-project/cache-hashed-abc123.zip to /tmp/test-project/cache-unhashed.zip
WARNING: processPath: artifact path is not a subpath of project directory: /tmp/test-project/some-dir

After:

$ ls -l /tmp/test-project
total 8
-rw-r--r--@ 1 taka  wheel  197  3月 23 13:47 cache-unhashed.zip
drwxr-xr-x@ 3 taka  wheel   96  3月 23 13:47 some-dir

Test 2: Build runner and connect with Gitlab.com

Build runner app and run it locally. Register it to Gitlab.com and confirm it works.

What are the relevant issue numbers?

Part of https://gitlab.com/gitlab-org/gitlab-runner/-/work_items/39007+.

Edited by Taka Nishida

Merge request reports

Loading