Skip to content

Geo: File synchronization concurrency limit actually limits jobs by half of the setting

Summary

If you set File synchronization concurrency limit to X, then concurrent Sidekiq jobs which sync "files" are limited to X/2.

Steps to reproduce

  1. For a Geo secondary site, set File synchronization concurrency limit to 4.
  2. Make Geo resync many files. In Rails console in a Puma/Sidekiq/Geo Log Cursor node in the secondary site:
    registry_entries = Geo::UploadRegistry.synced.limit(50)
    registry_entries.each { |r| r.pending! }
  3. Look at Sidekiq logs.

You should see that Geo::EventWorker jobs with args { 'upload', :created, model_record_id: XXXX } are only scheduled such that 2 are running at any time.

(Note: It may go above that if new uploads are being added during the observed time... This is a separate problem with our concurrency limiting).

Example Project

What is the current bug behavior?

Geo::EventWorker jobs with args { 'upload', :created, model_record_id: XXXX } are only scheduled such that a maximum of File synchronization concurrency limit/2 are running at any time.

What is the expected correct behavior?

Geo::EventWorker jobs with args { 'upload', :created, model_record_id: XXXX } are only scheduled such that a maximum of File synchronization concurrency limit are running at any time.

Relevant logs and/or screenshots

Output of checks

Results of GitLab environment info

Expand for output related to GitLab environment info

(For installations with omnibus-gitlab package run and paste the output of:
`sudo gitlab-rake gitlab:env:info`)

(For installations from source run and paste the output of:
`sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)

Results of GitLab application Check

Expand for output related to the GitLab application check

(For installations with omnibus-gitlab package run and paste the output of: sudo gitlab-rake gitlab:check SANITIZE=true)

(For installations from source run and paste the output of: sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)

(we will only investigate if the tests are passing)

Possible fixes