Skip to content
Snippets Groups Projects

Change data consistency for workers maintained by importers

What does this MR

It updates workers data consistency from :always to :sticky for workers maintained by importers, as a way to reduce database reads on the primary DB. Check #462611 (closed).

To reduce resource saturation on the primary node, all workers should be changed to sticky, at minimum.

Since jobs are now enqueued along with the current database LSN, the replica (for :sticky or :delayed) is guaranteed to be caught up to that point, or the job will be retried, or use the primary. Consider updating the worker(s) to delayed, if it's applicable.

You can read more about the Sidekiq Workers data_consistency in https://docs.gitlab.com/ee/development/sidekiq/worker_attributes.html#job-data-consistency-strategies.

You can use this dashboard to monitor the worker query activity on primary vs. replicas.

Currently, the gitlab-housekeeper is not always capable of updating all references, so you must check the diff and pipeline failures to confirm if there are any issues.

This change was generated by gitlab-housekeeper using the Keeps::UpdateWorkersDataConsistency keep.

To provide feedback on your experience with gitlab-housekeeper please create an issue with the label GitLab Housekeeper and consider pinging the author of this keep.

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
Please register or sign in to reply
Loading