Skip to content

Refactor pipeline creation service

Kamil Trzciński requested to merge refactor-builds-creation-service into master

What does this MR do?

This refactors GitLab CI build processing: all builds for pipeline are pre-created when a pipeline object is created. The builds are created with a new introduced status created. The builds are then automatically promoted to pending when a previous stage do succeed. This significantly simplifies pipeline processing code solving a lot of problems of lazily initialisation of previous approach (builds were created on-demand).

Why was this MR needed?

The previous mechanism had a lot of flows (shown in related issues) in how it work, but also in code design. Removing cross model-service-library dependencies.

The current approach moves a build creation to single place CreatePipelineService and removes a dynamic dependency on config_processor significantly simplifying a build creation and pipeline processing. Pipeline processing is implemented in ProcessPipelineService.

This also allows to easily extend GitLab with Manual Actions which is part of 8.10 direction issue.

Migration problem

This MR removes the a on-demand creation of builds in pipelines. Pipelines that are running and are in mid-stage (some stages started, but not all) will not be fully evaluated after application restart. This happens, because the code responsible for on-demand creation is removed. There's no easy way to migrate existing pipelines, other than doing offline migration and putting pipeline processing in migration code (which seems to be a really bad idea).

To support old pipelines I added a lazy initialization of builds if none is found.

What are the relevant issue numbers?

Fixes: https://gitlab.com/gitlab-org/gitlab-ce/issues/12839 Solves: https://gitlab.com/gitlab-org/gitlab-ce/issues/18644 https://gitlab.com/gitlab-org/gitlab-ci-multi-runner/issues/289 Allows to easily implement: https://gitlab.com/gitlab-org/gitlab-ce/issues/17010

Does this MR meet the acceptance criteria?

Merge request reports