Skip to content

Update Migration::EnqueuerWorker capacity logic

Steve Abrams requested to merge 357094-fix-capacity-phase2 into master

🍃 Context

We are preparing to migrate all container repositories to the new container registry. Rails is responsible for driving this migration by using an EnqueuerWorker to find the next qualified container repository to import and making an API request to the registry to begin the import.

We have a variety of feature flags and settings to allow us to control the speed of the imports. One thing we control is the "capacity", or the number of imports we allow to be in progress at one time. For example, we can set the capacity to 10. If an EnqueuerWorker job is executing, and sees there are already 10 imports in progress, it will not schedule any more jobs. If there are less than 10 imports in progress, the EnqueuerWorker will schedule another job along the way.

The EnqueuerWorker checks the capacity twice:

  1. At the beginning of #perform to see if it should even proceed with kicking off an import.
  2. At the end of #perform to see if it should re-enqueue another Enqueuer job.

In testing, we realized that in the time between (1.) and (2.), the current_capacity (imports in progress) can change. This led us to realizing we were incorrectly memoizing the current_capacity and along the way, we noticed that we had inconsistent capacity checks, using < in one place, and <= in another place. This is what #357094 (closed) describes.

💭 What does this MR do and why?

This MR removes the strong_memoize of current_capacity and updates so all capacity checks correctly use current <= maximum_capacity.

The import functionality is all behind a feature flag and we will be specifically testing this behavior on staging for the next few weeks.

🖼 Screenshots or screen recordings

See below.

💻 How to set up and validate locally

Trying to change the current_capacity while the job is running locally is not easy. The best way I've found to do this is with pry, inserting binding.pry in the #below_capacity? method, and then creating additional importing container repositories in the middle of a test:

FactoryBot.create(:container_repository, :importing)

Using any of the tests that use the 're-enqueuing based on capacity' shared example will test the capacity:

bin/rspec ./spec/workers/container_registry/migration/enqueuer_worker_spec.rb:77
bin/rspec ./spec/workers/container_registry/migration/enqueuer_worker_spec.rb:149
bin/rspec ./spec/workers/container_registry/migration/enqueuer_worker_spec.rb:205

📏 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #357094 (closed)

Edited by Steve Abrams

Merge request reports