Unable to deploy an auto-deploy version that only had CNG changes
Problem
We have a logical flaw in how we record metadata. A gitlab package is always complete, if something doesn't change from the previous tag it still has to be recorded in metadata.
But in release-tools, we defer metadata recording to tagger classes, and this will result in partial information is something doesn't change.
Failure scenario
Today we added https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/1316 to gitlab-rails in order to pull the mimemagic
gem directly from GitHub.
This failed while building CNG images, because the Dockerfile it uses didn't have git
available.
DJ added git support in gitlab-org/build/CNG!629 (merged), which got merged and cherry-picked into https://gitlab.com/gitlab-org/security/charts/components/images/commits/13-11-auto-deploy-2021032418.
I then manually ran auto_deploy:tag
to bring in the change. This worked, until we got to the auto_deploy:coordinated:deploy
job: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/3461003
The job failed because the metadata file didn't have an omnibus-gitlab-ee
entry available, because there was no Omnibus change during this tag -- it was purely a CNG change.
The metadata file initially looked like this: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/ed492c9ee5a41d0ca10b18aa91f3a8ee47e50a39
I manually edited it to bring in the entry from the prior auto-deploy tagging: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/75eb96176effd7d3263c9ab09d8889aa5bc61a46.
That got the wrong data somehow (I blame the GitLab web editor interface), so I fixed it locally: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/4cd0ab4338129d2420475ed8b38af159974df0b5
The deployer trigger depends on the Omnibus tag here: https://gitlab.com/gitlab-org/release-tools/blob/fc83fee7cf7c4c1ce2b8268ed50f94009d948e30/lib/release_tools/tasks/auto_deploy/deploy_trigger.rb#L68
Solution
Pre-populating metadata has resolved this issue.
- CNG-only change: https://gitlab.com/gitlab-org/security/charts/components/images/-/commit/8144bb33eaa196a2c965b457b428f7ee83f1c573
- Tagging: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/3957749
- Resulting metadata file, including
omnibus-gitlab-ee
from the previous tag: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/80a8d4f509b55fd37a62c966b6ed23a95d72a23d