Merging security MRs need to log what is merged, and be more explicit
the chatops command
/chatops run release merge --security --master
is opaque:
It displays the result of the merge but the job doesn't show you what it did: https://gitlab.com/gitlab-org/release-tools/-/jobs/260116047
This has contributed to a few issues recently:
- For release 11.10.3, 11.9.11, 11.8.10 we ended up merging a security MR into that wasn't intended for the release gitlab-org/release/retrospectives#20 (closed). This was noticed very late into the process and it forced us to create a new point release for 11.10
- For release 12.1.2, 12.0.4, 11.11.6 we had the same issue, where a developer closed the 11.11 MR thinking that the deadline was missed, this resulted in only two backports being merged, which forced us to bump the security release. Luckily we noticed this before it was released. https://gitlab.com/gitlab-org/release/retrospectives/issues/26
- In another case a developer created more backports than necessary, this was automatically merged by the bot which resulted in a commit on the
11-10
release that meant that there was a commit on dev that was never released into a package. We needed to revert this commit (or simply bring it over to gitlab.com)
All three of these cases were because the chatops command that merges security releases makes assumptions about what is merged and trusts that checklists are followed and that all MRs for backports and master are treated as a single unit.
I think we have been putting off improvements because when we move the security releases to gitlab.com we can better utilize associated MRs to the security issue. Instead of labels and assigning to the bot perhaps we should have a yaml file that explicitly lists all the MRs for the security release that we use as the source of truth for both the chatops command and the security issue?