Skip to content
Snippets Groups Projects

Wednesday 2024-04-24 13:44 UTC - `gitlab-org/gitlab` broken `16-11-stable-ee` with update-gitaly-binaries-cache-fips, update-gitaly-binaries-cache, setup-test-env-fips, setup-test-env

    • Closed Incident created by 🤖 GitLab Bot 🤖 @gitlab-bot

    gitlab-org/gitlab pipeline #1266074426 failed

    Pipeline ID Branch Commit Merge request Source Duration Triggered by
    1266074426 16-11-stable-ee Merge remote-tracking branch 'dev/16-11-stable-ee' into 16-11-stable-ee N/A push 39.1 minutes GitLab Release Tools Bot

    Failed jobs (4):

    General guidelines

    A broken stable branch prevents patch releases from being built. Fixing the pipeline is a priority to prevent any delays in releases.

    The process in the Broken master handbook guide can be referenced since much of that process also applies here.

    Investigation

    Be sure to fill the Timeline for this incident.

    1. If the failure is new, and looks like a potential flaky failure, you can retry the failing job. Make sure to mention the retry in the Timeline and leave a link to the retried job.
    2. Search for similar master-broken issues in https://gitlab.com/gitlab-org/quality/engineering-productivity/master-broken-incidents/-/issues
    3. If one exists, ask the DRI of the master-broken issue to cherry-pick any resulting merge requests into the stable branch

    @gitlab-org/release/managers if the merge request author or maintainer is not available, this can be escalated using the dev-on-call process in the #dev-escalation slack channel.

    Pre-resolution

    If you believe that there's an easy resolution by either:

    • Reverting a particular merge request.
    • Making a quick fix (for example, one line or a few similar simple changes in a few lines). You can create a merge request, assign to any available maintainer, and ping people that were involved/related to the introduction of the failure. Additionally, a message can be posted in #backend_maintainers or #frontend_maintainers to get a maintainer take a look at the fix ASAP.
    • Cherry picking a change that was used to fix a similar master-broken issue.

    Resolution

    Add a comment to this issue describing how this incident could have been prevented earlier in the Merge Request pipeline (rather than the merge commit pipeline).

    Child items 0

    • No child items are currently assigned. Use child items to break down this issue into smaller parts.

    Activity

    Please register or sign in to reply