Geo: File synchronization concurrency limit actually limits jobs by half of the setting
Summary
If you set File synchronization concurrency limit
to X, then concurrent Sidekiq jobs which sync "files" are limited to X/2.
Steps to reproduce
- For a Geo secondary site, set
File synchronization concurrency limit
to 4. - Make Geo resync many files. In Rails console in a Puma/Sidekiq/Geo Log Cursor node in the secondary site:
registry_entries = Geo::UploadRegistry.synced.limit(50) registry_entries.each { |r| r.pending! }
- Look at Sidekiq logs.
You should see that Geo::EventWorker
jobs with args { 'upload', :created, model_record_id: XXXX }
are only scheduled such that 2 are running at any time.
(Note: It may go above that if new uploads are being added during the observed time... This is a separate problem with our concurrency limiting).
Example Project
What is the current bug behavior?
Geo::EventWorker
jobs with args { 'upload', :created, model_record_id: XXXX }
are only scheduled such that a maximum of File synchronization concurrency limit
/2 are running at any time.
What is the expected correct behavior?
Geo::EventWorker
jobs with args { 'upload', :created, model_record_id: XXXX }
are only scheduled such that a maximum of File synchronization concurrency limit
are running at any time.
Relevant logs and/or screenshots
Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true
)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true
)(we will only investigate if the tests are passing)