Skip to content

Extend job archival mechanism to the whole pipeline

Context

We currently have a bunch of mechanisms implemented that allow us to "degenerate" a build and to "archive" those that seem to be outdated. We are not using these mechanisms and we should revisit them as part of the &4685 (closed) initiative. We should figure it out what is a difference between a degenerated and archived build, and figure the right cadence of degenerating builds.

Problem

Build archival only applies to Ci::Builds today. GenericCommitStatus and Ci::Bridge jobs are not affected by it. For this reason we don’t have the concept of “pipeline archived”.

We can have a pipeline with archived builds today but still allow GenericCommitStatus jobs to be added using the commit status API. Or we would still allow Ci::Bridge jobs to be retried.

One of these problems is described in this issue: Unclear behavior in UI when clicking "Retry" fo... (#408796 - closed).

We need to establish a SSoT definition that applies to the whole pipeline.

There are 2 different definitions of "archived"

  • A build is archived when older than the threshold in the application settings (and if the threshold is set)
  • A pipeline (as per this design doc) is considered archived when moved to a non-current partition.

The 2 definitions don't always align, not even if the archive_builds_older_than setting is equivalent to the time a partition remains "current".

Details from discussions

The definition of "archived" must be defined at pipeline level in order to be consistent with all the jobs in a pipeline. We also need to enforce this invariant when jobs are retried, or manual jobs are played, or external jobs are appended, etc.

One challenge with this is that we would need to encode this invariant (check if the pipeline is archived) in all service objects that affect the pipeline or its jobs (retry pipeline, retry any type of job, play any type of manual job, etc.)

One idea could be to encode these invariants in a single place and maybe have a general service object Ci::PipelineService that allows users to perform specific actions.

service = Ci::PipelineService.new(current_user: ..., pipeline: ...)
service.add_job(...)
service.retry_job(...) # enforce pipeline archived and then call Ci::RetryJobService.new(...) under the hood.

Rollout

Edited by Cheryl Li