artifacts are removed for the most recent successful pipeline

Summary

Occasionally artifacts are removed for the most recent successful pipeline, despite Keep artifacts from most recent successful jobs being enabled.

Steps to reproduce

This is a transient problem. Most of the time it works correctly. We are not clear on how to produce it, nor have we noticed a pattern when it occurs.

Example Project

What is the current bug behavior?

We use curl to download the most recent successful artifacts for a pipeline. Sometimes this fails:

curl -k -f -sS --retry 7 -o swoop-core-base-package.artifact --header 'PRIVATE-TOKEN: [MASKED]' 'https://gitlab.swoop.local/api/v4/projects/93/jobs/artifacts/master/download?job=swoop-core-base-package'
08:11:00 curl: (22) The requested URL returned error: 404

When I dig into this, I find the artifacts have been removed, even though they appeared to be the most recent successful build at the time.

What is the expected correct behavior?

The artifacts for the most recent successful build should not be removed, and the curl should succeed.

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

We are running gitlab locally, version 17.5. I have slack comments showing the problem hit 6 months ago on older versions as well.

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

swoopadmin@gitlab:~$ sudo gitlab-rake gitlab:env:info

(removed, can be provided in confidential environment.)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)
swoopadmin@gitlab:~$ sudo gitlab-rake gitlab:check SANITIZE=true
(removed, can be provided in confidential environment.)

Possible fixes