[Feature flag] Rollout of `geo_package_file_verification`

What

Remove the :geo_package_file_verification feature flag ...

Owners

• Team: groupgeo
• Most appropriate slack channel to reach out to: #g_geo
• Best individual to reach out to: @mkozono

Expectations

What are we expecting to happen?

• Geo primary begins backfilling the verification_checksum field on packages_package_files
• Geo primary begins scheduling Geo::VerificationWorker to fill verification_checksum when a PackageFile is created
• Geo secondaries begin backfilling the verification_checksum field on package_file_registry
• Non-Geo deployment, nothing

What might happen if this goes wrong?

• Verification job exceptions

What can we monitor to detect problems with this?

Sentry errors from classes like "Geo::VerificationWorker" or "Geo::VerificationBatchWorker"

Beta groups/projects

If applicable, any groups/projects that are happy to have this feature turned on early. Some organizations may wish to test big changes they are interested in with a small subset of users ahead of time for example.

• N/A

Roll Out Steps

• Enable on staging (/chatops run feature set geo_package_file_verification true --staging)
• Test on staging
• Ensure that documentation has been updated => !54361 (merged)
• Double check that enabling the feature does not affect non-Geo deployments
• [-] Enable on GitLab.com for individual groups/projects listed above and verify behaviour (/chatops run feature set --project=gitlab-org/gitlab feature_name true)
• [-] Coordinate a time to enable the flag with #production and #g_delivery on slack.
• [-] Announce on the issue an estimated time this will be enabled on GitLab.com
• [-] Enable on GitLab.com by running chatops command in #production (/chatops run feature set feature_name true)
• [-] Cross post chatops Slack command to #support_gitlab-com (more guidance when this is necessary in the dev docs) and in your team channel
• [-] Announce on the issue that the flag has been enabled
• Test on GCP (e.g. use ansible-gitlab-geo to setup deployment, seed package files, enable FF, check statuses on Admin > Geo > Nodes)
• Enable feature flag by default and add changelog entry => !54361 (merged)
• Remove feature flag
• [-] After the flag removal is deployed, clean up the feature flag by running chatops command in #production channel
• After the flag removal is deployed, clean up the feature flag by running chatops command in #staging channel