Skip to content

Create issue in release/tasks for broken stable

Steve Abrams requested to merge delivery2782-stable-branch-failure-ci-issue into master

⚠ Before this is merged, we need to add the release tools project token to the BROKEN_BRANCH_INCIDENTS_PROJECT_TOKEN environment variable in gitlab.com/gitlab-org/gitlab (see https://gitlab.com/gitlab-com/gl-infra/delivery/-/issues/2815). I will remove this note once it has been added.

Rémy: I created a project token at https://gitlab.com/gitlab-org/release/tasks/-/settings/access_tokens and created the BROKEN_STABLE_INCIDENTS_PROJECT_TOKEN variable in https://gitlab.com/gitlab-org/gitlab/-/settings/ci_cd. ✅

🌳 What does this MR do and why?

Updates the GitLab CI to automatically generate issues in https://gitlab.com/gitlab-org/release/tasks/-/issues when a stable branch experiences a pipeline failure. This is the same behavior we currently have for master-broken, but we are replicating it for broken stable branches.

📽 Screenshots or screen recordings

This is an example of an issue generated by the script: gitlab-org/release/tasks#4952 (closed)

💻 How to set up and validate locally

This is not easily tested since it depends on a pipeline failure on a commit targeting a stable branch. We already know the notify-pipeline-failure job runs successfully when a stable branch fails, so here we can focus purely on testing if the scripts/create-pipeline-failure-incident.rb will generate the expected issue. I was able to test this by gathering a variety of ENV variables to run it locally, but against real projects:

CI_COMMIT_REF_NAME=15-6-stable-ee \
CI_COMMIT_SHA=bfcd2b9b5cad0b889494ce830697392c8ca11257 \
CI_PROJECT_PATH=gitlab.com/gitlab-org/gitlab-foss \
CI_COMMIT_REF_NAME=15-6-stable-ee \
CI_PROJECT_NAME=gitlab \
GITLAB_USER_ID=4059254 \
CI_PROJECT_ID=13083 \
CI_PIPELINE_ID=756644007 \
CI_PIPELINE_URL=https://gitlab.com/gitlab-org/gitlab-foss/-/pipelines/756644007 \
CI_PROJECT_URL=https://gitlab.com/gitlab-org/gitlab-foss \
CI_PIPELINE_CREATED_AT=2023-01-24\ 00:00:00 \
CI_COMMIT_TITLE=Add\ latest\ changes\ from\ gitlab-org/gitlab@15-6-stable-ee \
CI_PIPELINE_SOURCE=push \
GITLAB_USER_NAME=GitLab\ Bot \
PROJECT_TOKEN_FOR_CI_SCRIPTS_API_USAGE=<token-from-gitlab-project> \
CI_SERVER_URL=https://gitlab.com \
GITLAB_USER_LOGIN=sabrams \
ruby create-pipeline-failure-incident.rb -p 5064907 -f incident.json -t <token-from-release-tasks-project>

Note this depends on two tokens that most users won't have access to, so it's unlikely you will be able to reproduce this yourself.

This test was made using this failing pipeline: https://gitlab.com/gitlab-org/gitlab-foss/-/pipelines/756644007, which resulted in this slack message: https://gitlab.slack.com/archives/C0XM5UU6B/p1674604244390159. With this update, the button in that message will point to this created issue.

🛃 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to gitlab-com/gl-infra/delivery#2782 (closed)

Edited by Rémy Coutable

Merge request reports