Geo: Remote stored Packages files should account for Object Storage settings

Summary

On 13.1 on staging.gitlab.com, all PackageFileRegistrys have this last_sync_failure: Skipping transfer as this secondary node is not allowed to replicate content on Object Storage. And they are all failed, and they keep getting retried but will never succeed.

Steps to reproduce

• Configure Package Registry to use remote storage on both Geo primary and secondary
• Leave this disabled for the Geo secondary Allow this secondary node to replicate content on Object Storage
• Create a Package File

Side note for anyone wanting to test package sync without using a seeder rake task (FILTER=packages bundle exec rake db:seed_fu):

• The Package Registry feature is enabled by default in GitLab, so you don't need to do admin stuff
• So you can locally clone any random repo of an NPM package, e.g. https://gitlab.com/gitlab-org/visual-review-tools
• Then follow https://docs.gitlab.com/ee/user/packages/npm_registry/index.html#getting-started
  • You already have npm and yarn installed for the GDK
  • If you are not sure what exactly to replace @foo with, look at https://docs.gitlab.com/ee/user/packages/npm_registry/index.html#package-naming-convention. In my case, I created my test project in a group called test, so I replaced @foo with @test. 🤷
• I published more files by bumping the version in package.json (I also edited a file but I think it's probably not necessary) and npm publish again

What is the current bug behavior?

BlobDownloadService runs for the new package file, and it exits with a failure because it fails preconditions. The package file shows up in Admin Area > Geo > Nodes Package Files progress bar counted as failed.

What is the expected correct behavior?

BlobDownloadService should never run for the package file. The package file should not show up in the Admin Area > Geo > Nodes Package Files progress bar (there should be no registry for the package file).

Relevant logs and/or screenshots

(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's tough to read otherwise.)

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Possible fixes

When Allow this secondary node to replicate content on Object Storage is disabled, a secondary should

• not process create events for object stored package files https://gitlab.com/gitlab-org/gitlab/blob/v13.1.0-ee/ee/app/models/concerns/geo/blob_replicator_strategy.rb#L24
• not create registries for object stored package files https://gitlab.com/gitlab-org/gitlab/-/blob/v13.1.0-ee/ee/app/models/packages/package_file.rb#L43-60