Unable to deploy an auto-deploy version that only had CNG changes

Problem

We have a logical flaw in how we record metadata. A gitlab package is always complete, if something doesn't change from the previous tag it still has to be recorded in metadata.

But in release-tools, we defer metadata recording to tagger classes, and this will result in partial information is something doesn't change.

Failure scenario

Today we added https://gitlab.com/gitlab-org/security/gitlab/-/merge_requests/1316 to gitlab-rails in order to pull the mimemagic gem directly from GitHub.

This failed while building CNG images, because the Dockerfile it uses didn't have git available.

DJ added git support in gitlab-org/build/CNG!629 (merged), which got merged and cherry-picked into https://gitlab.com/gitlab-org/security/charts/components/images/commits/13-11-auto-deploy-2021032418.

I then manually ran auto_deploy:tag to bring in the change. This worked, until we got to the auto_deploy:coordinated:deploy job: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/3461003

The job failed because the metadata file didn't have an omnibus-gitlab-ee entry available, because there was no Omnibus change during this tag -- it was purely a CNG change.

The metadata file initially looked like this: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/ed492c9ee5a41d0ca10b18aa91f3a8ee47e50a39

I manually edited it to bring in the entry from the prior auto-deploy tagging: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/75eb96176effd7d3263c9ab09d8889aa5bc61a46.

That got the wrong data somehow (I blame the GitLab web editor interface), so I fixed it locally: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/4cd0ab4338129d2420475ed8b38af159974df0b5

The deployer trigger depends on the Omnibus tag here: https://gitlab.com/gitlab-org/release-tools/blob/fc83fee7cf7c4c1ce2b8268ed50f94009d948e30/lib/release_tools/tasks/auto_deploy/deploy_trigger.rb#L68

Solution

Pre-populating metadata has resolved this issue.

1. CNG-only change: https://gitlab.com/gitlab-org/security/charts/components/images/-/commit/8144bb33eaa196a2c965b457b428f7ee83f1c573
2. Tagging: https://ops.gitlab.net/gitlab-org/release/tools/-/jobs/3957749
3. Resulting metadata file, including omnibus-gitlab-ee from the previous tag: https://ops.gitlab.net/gitlab-org/release/metadata/-/commit/80a8d4f509b55fd37a62c966b6ed23a95d72a23d