Add verification step to ensure all packages are released

Someone asked where the 11.0.6 packages for OpenSUSE were in https://about.gitlab.com/2018/08/28/security-release-gitlab-11-dot-2-dot-2-released/#comment-4075180454, and I found that some of these manual steps in https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/2769663 had not been triggered:

When I try to trigger them, I got:

I'm not sure how this was missed, but I assume the ChatOps script may have run into some error?

/cc: @rspeicher, @jivanvl

changed the description

I can't seem to retrigger these steps.

@ayufan What does "There has been a missing dependency failure" even mean?

It looks related to expired artifacts, although it's interesting to me that the feature flag isn't enabled?

irb(main):016:0> Feature.enabled?('ci_disable_validates_dependencies')
=> false
irb(main):017:0> job.dependencies.all?(&:valid_dependency?)
=> false
irb(main):018:0> job.dependencies.first.artifacts_expired?
=> true

Will we need to rebuild these?

Yeah, it looks like I had to rebuild OpenSUSE for this step to work again. I triggered a rebuild on the SLES and CentOS 7 builds as well.

added to epic &9 (closed)

As mentioned in gitlab-org/omnibus-gitlab#3878 (closed), our PackageCloud upload for CentOS 7 CE failed due to a DNS error: https://dev.gitlab.org/gitlab/omnibus-gitlab/-/jobs/3249409

This is particularly a problem with security releases because there may be a critical fix that needs to be available when we post.

In the past, I've done spot checks that the number of packages for EE and CE were right (e.g. https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=11.4.3 matches https://packages.gitlab.com/app/gitlab/gitlab-ce/search?q=11.4.0) before merging the blog post. I think we should be able to use the PackageCloud API (or even a direct curl) to check that all the packages have actually made it to the server.

Issue https://gitlab.com/gitlab-org/release/framework/issues/209 has been created to address this. I will close this one.

closed