Keep latest artifacts for the last successful jobs
### Problem to Solve The `expire_in` function is great, but depending on how often your pipeline is built can be hard to tune. If you don't build frequently, setting a short `expire_in` can result in your most recent artifact being deleted. Setting it too long can keep unneeded copies around. Over time this can use up a lot of disk space. Most of the time, people just want to ensure they always at least have the most recent artifact, whenever that was built. ### Proposal Implement a default behavior for all projects to always keep the latest artifact for each successful job that was part of a successful pipeline (that produced artifacts) on any active (non-deleted) branch, merge request, or tag (i.e., the one downloaded via https://docs.gitlab.com/ce/user/project/pipelines/job_artifacts.html#downloading-the-latest-artifacts is exempt from any `expire_in` policy). In other words, **never delete** the most recent artifact (even if expired), which is same artifact as described at the docs link here: ![image](/uploads/c29013d1f233f16e300915dd76902108/image.png) Additionally: - Artifacts for deleted branches follow `expire_in` - All non-latest artifacts follow `expire_in` - Only the latest artifact for a job on a non-deleted branch is kept This would allow for setting a relatively aggressive expiration policy, while remaining confident you always have your most recent artifact for any active branches. #### Implementation notes Because `expire_in:` could be set too short we would need to define a way to "lock" the latest artifacts to prevent deletion by `ExpireBuildArtifactsWorker`. For example add a new column `job_artifacts.locked` (false by default). Then when a pipeline completes we will lock the latest artifacts in a moving window: lock last ones and unlock anything from previous pipelines. Finally the `ExpireBuildArtifactsWorker` would need to be modified to filter out locked artifacts. Another aspect to consider is to unlock artifacts when a branch is deleted or they would remain locked and never deleted. #### Out of Scope for this MVC Items not expected for this MVC issue: * ability to keep latest **N** count of artifacts per job * new checkbox in project's `Settings > CI/CD > General pipelines` for `Keep latest job artifacts per ref` * allow artifacts to live longer than **N** days for compliance reasons * aggressively remove artifacts which are not the latest even before they expire
issue