Cache: select newer remote artifact when FF_HASH_CACHE_KEYS is toggled
What does this MR do?
When FF_HASH_CACHE_KEYS is toggled, the primary cache key changes between hashed and
unhashed forms. Without this change, the first run after toggling always results in a cache
miss, because the cache under the new key does not exist yet.
This MR adds alternate URL support to the cache extractor. On each download, the runner now compares the timestamps of the primary and alternate cache objects and picks the newer one. This means the first run after toggling the FF can still get a cache hit by falling back to the artifact stored under the old key.
Related MR: Cache: rename local artifact when FF_HASH_CACHE... (!6546 - merged)
Why was this MR needed?
To support smooth transition when feature flag is toggled.
What's the best way to test this MR?
Tested locally with a shell executor runner connected to GitLab.com, using a GCS cache backend.
| # | FF_HASH_CACHE_KEYS | Remote state | Download (This MR) | Upload | Job (GCS) | Job (AWS S3) |
|---|---|---|---|---|---|---|
| 1 | off | empty | Miss (both URLs 404) | Uploads to unhashed key | job | job |
| 2 | off | unhashed exists | Hit (primary selected) | Skips upload (already exists) | job | job |
| 3 | off → on | unhashed exists, hashed missing | Hit (alternate selected: unhashed is newer) | Uploads to hashed key (migration) | job | job |
| 4 | on | both exist, hashed newer | Hit (primary selected) | Skips upload (already exists) | job | job |
| 5 | on → off | hashed exists, unhashed missing | Hit (alternate selected: unhashed not available) | Uploads to unhashed key (reverse migration) | job | job |
The key scenario is Scenario No. 3: the first run after enabling the FF gets a cache hit by falling back to the unhashed artifact, then migrates it to the hashed key in the same job.