Skip to content

Adds data consistency for Elastic::BulkCronWorker(s) [RUN ALL RSPEC] [RUN AS-IF-FOSS]

Dmitry Gruzd requested to merge 326719-switch-bulk-cron-workers-to-replicas into master

What does this MR do?

Utilize new Sidekiq read-only database replicas capabilities (introduced in !55881 (merged)) for ElasticIndexBulkCronWorker and ElasticIndexInitialBulkCronWorker.

Data consistency is set to :delayed - worker would switch to primary only on write, would use replica always if there's a long replication lag the job will be delayed, and only if the replica is not up to date on the next retry, it will switch to the primary.

This is set behind the feature flag: load_balancing_for_bulk_cron_workers and it will be disabled by default

#326719 (closed)

Relates to #322452 (closed)

Relates to gitlab-org/search-team/team-tasks#36 (closed)

Screenshots (strongly suggested)

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • [-] Label as security and @ mention @gitlab-com/gl-security/appsec
  • [-] The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • [-] Security reports checked/validated by a reviewer from the AppSec team

Related to #326719 (closed)

Edited by Dmitry Gruzd

Merge request reports