Skip to content

Disable JobWaiter when there are many jobs

Sean McGivern requested to merge disable-jobwaiter-for-many-jobs into master

What does this MR do?

In WaitableWorker, when there are fewer than four jobs to process we simply process them without going through Sidekiq. When there are more jobs, we run them in Sidekiq but add a second argument for a JobWaiter key, and wait for timeout seconds to see if those jobs finished.

When there are very many jobs in the batch, we will never finish the batch before the timeout. This has these consequences:

  1. The HTTP request takes longer as we're waiting for the timeout.
  2. The jobs can't be deduplicated as the second argument is a unique JobWaiter key.

This change adds a feature flag, skip_job_waiter_for_large_batches, that doesn't use the JobWaiter key when the batch size is more than ten times the timeout, as we can process approximately ten jobs per second. By default the flag is off.

For gitlab-com/gl-infra/scalability#205 (closed).

Merge request reports