Delete artifacts based on latest "job" not "pipeline"
Summary
Currently the artifacts for the latest pipeline of a given branch are not deleted (even when expired). The following text is shown:
These artifacts are the latest. They will not be deleted (even if expired) until newer artifacts are available.
This is fine, except that it apparently bases its "being the latest" test on "being the latest pipeline" for the current branch, whereas the logics would expect "being the latest same job".
In particular in GIMP, we have a weekly schedule producing a test installer for Windows. The job creating the installer is win-installer-nightly
. This is not run at every commit but on a schedule and we want people to always have access to the last version of the installer. Say the last scheduled job fail, we want people to be able to have access to the previous installer even if it expired (because it's still the last successful win-installer-nightly
job).
Steps to reproduce
- Create a job
test
with some artifacts but which runs only manually or through a schedule. - Run a pipeline with this job once.
- Now push more commits creating new pipelines without the job.
- Notice that the artifacts get removed when the expiration date happens.
Example Project
For instance this job: https://gitlab.gnome.org/GNOME/gimp/-/jobs/1322768
It says:
Job artifacts
The artifacts will be removed in 13 hours
Unfortunately there were no other job done. We can increase the retention period (which I'm going to do now), but it won't save us from the case when the schedule job would fail (we can't have huge retention period; we used to have bigger ones, but it filled our data limit and we had CI failures because of lack of space).
What is the current bug behavior?
The artifacts are removed when they expire even if this job has not been run again.
What is the expected correct behavior?
The artifacts should not be removed as long as:
- this same job is not run again and succeed (creating new artifacts);
- this job is not deleted from
gitlab-ci.yml
.
Relevant logs and/or screenshots
Output of checks
This happens on gitlab.gnome.org.
Results of GitLab environment info
Whatever is installed on the GNOME instance of Gitlab.
Results of GitLab application Check
N/A
Possible fixes
Artifact retention should check the expire_in
value of course but also if:
- The job is not the last of a given name for the current branch (not if the pipeline is the last);
- Or if no job with this name exists anymore in the gitlab-ci.yml (in which case, the artifacts can be removed safely too).