Skip to content

Ensure Reenqueuer workers are not deduplicated

Michael Kozono requested to merge mk/reenqueuer-jobs-are-not-idempotent into master

What does this MR do?

In a strict sense, RegistryConsistencyWorker (which leverages Reenqueuer) is not idempotent, since a subsequent run may alter the system state. It meets criteria 1 and 2 below, but not the 3rd one (note, I modified the 3rd criteria in this MR to more accurately match the codebase.)

From https://gitlab.com/gitlab-org/gitlab/-/blob/v13.6.1-ee/rubocop/cop/scalability/idempotent_worker.rb:

Avoid adding not idempotent workers.

A worker is considered idempotent if:

  1. It can safely run multiple times with the same arguments
  2. The application side-effects are expected to happen once (or side-effects of a second run are not impactful)
  3. It can safely be skipped if another job with the same arguments is already running or is in the queue

If all the above is true, make sure to mark it as so by calling the idempotent! method in the worker scope.

See https://github.com/mperham/sidekiq/wiki/Best-Practices#2-make-your-job-idempotent-and-transactional

This is generalizable to any worker that leverages Reenqueuer (there are no others that do yet).

Resolves #284826 (closed)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Michael Kozono

Merge request reports

Loading