Automatically check deployed versions for self-managed releases
Related to #1176 (closed) and the problems we've seen with a security release in gitlab-com/www-gitlab-com!61470 (merged), gitlab-com/www-gitlab-com!61493 (merged), gitlab-com/www-gitlab-com!61514 (merged) .
One of the tasks we have in our release check list is to manually confirm whether the version on the help page of the deploy instance matches the one that we expect to install. As we've seen, it is trivial to get distracted and not complete that task correctly.
Proposal
Add to release-finish and pre-finish a job that would query the API of the deployment instance and compare the versions of the package that got deployed to the version that the application is reporting.
In case of the same version, take no further action. In case of a failure, trigger a slack notification with @ mention of release-managers
to inform that that something needs to be investigated.
@gitlab-org/delivery What are you thoughts on this?