Skip to content

Delete artifacts based on latest "job" not "pipeline"

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

Currently the artifacts for the latest pipeline of a given branch are not deleted (even when expired). The following text is shown:

These artifacts are the latest. They will not be deleted (even if expired) until newer artifacts are available.

This is fine, except that it apparently bases its "being the latest" test on "being the latest pipeline" for the current branch, whereas the logics would expect "being the latest same job".

In particular in GIMP, we have a weekly schedule producing a test installer for Windows. The job creating the installer is win-installer-nightly. This is not run at every commit but on a schedule and we want people to always have access to the last version of the installer. Say the last scheduled job fail, we want people to be able to have access to the previous installer even if it expired (because it's still the last successful win-installer-nightly job).

Steps to reproduce

  • Create a job test with some artifacts but which runs only manually or through a schedule.
  • Run a pipeline with this job once.
  • Now push more commits creating new pipelines without the job.
  • Notice that the artifacts get removed when the expiration date happens.

Example Project

For instance this job: https://gitlab.gnome.org/GNOME/gimp/-/jobs/1322768

It says:

Job artifacts

The artifacts will be removed in 13 hours

Unfortunately there were no other job done. We can increase the retention period (which I'm going to do now), but it won't save us from the case when the schedule job would fail (we can't have huge retention period; we used to have bigger ones, but it filled our data limit and we had CI failures because of lack of space).

What is the current bug behavior?

The artifacts are removed when they expire even if this job has not been run again.

What is the expected correct behavior?

The artifacts should not be removed as long as:

  • this same job is not run again and succeed (creating new artifacts);
  • this job is not deleted from gitlab-ci.yml.

Relevant logs and/or screenshots

Output of checks

This happens on gitlab.gnome.org.

Results of GitLab environment info

Whatever is installed on the GNOME instance of Gitlab.

Results of GitLab application Check

N/A

Possible fixes

Artifact retention should check the expire_in value of course but also if:

  • The job is not the last of a given name for the current branch (not if the pipeline is the last);
  • Or if no job with this name exists anymore in the gitlab-ci.yml (in which case, the artifacts can be removed safely too).
Edited by 🤖 GitLab Bot 🤖