Asking for pipeline status contains a bug
❌
Problem Statement There are steps in our release procedures that ask us to request the status of our pipelines for various stable branches or tags at key moments. Example. On the 20th of February, we accomplished the tag of an RC for the upcoming 15.9 release. After tagging was done and the environment deployed too, a status of 15.9 was requested:
The provided information is incorrect. The output is only for the stable branches and it misses all the pipelines, and in this case, the tags. Specifically, the tag for our FOSS build was failing with a legit problem! Reference: https://gitlab.com/gitlab-org/gitlab-foss/-/jobs/3805512575
The code responsible for checking for status looks only at one page of builds and returns the status of only the first build in that list: https://gitlab.com/gitlab-org/release-tools/-/blob/58a9a0fd13bac06d1bdd9d2175171ded3b0f5fe7/lib/release_tools/branch_status.rb#L37-41
Which from a technical standpoint, is the incorrect job we should be evaluating! See the below:
Without fixing this, we risk missing legitimate QA failures prior to tagging GitLab. Tagging is very tough to recover from. In some cases, we completely abandon versions and move on to the next patch semver version. Recovering on a minor release would've been very tough. I'm marking this as high priority as the visibility into this is critical.
Resolution
Review the logic we currently leverage and build out a better notification that looks at either all pipelines or the more correct pipelines to ensure that we are correctly notified of the appropriate status.