Add exclusive lock on project in ElasticCommitIndexerWorker
What does this MR do?
Indexing the same project twice at the same time causes conflict errors in Elasticsearch and this usually leads to failures which can lead to a project never being successfully indexed correctly. The simplest solution is to use a lock in the worker and just fail immediately if it's already running and let Sideikiq retries requeue the project if it does have more updates that need indexing. When it next runs it will check if there are new commits that need indexing.
We use a 1 hour TTL on the lock since it can take a long time to index a new project.
NOTE: We do de-duplicate this job in-queue as well but this de-duplication key is cleared (rightly so) when the job starts so long running jobs will still end up running duplicates in parallel and that's why we need this extra lock.
Screenshots
Does this MR meet the acceptance criteria?
Conformity
-
Changelog entry - [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec -
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Closes #32648 (closed)