Skip to content
Snippets Groups Projects
Commit cd5f35ff authored by Leonardo da Rosa's avatar Leonardo da Rosa :one:
Browse files

Change data consistency for workers maintained by not_owned

## What does this MR

It updates workers data consistency from `:always` to `:sticky` for workers maintained by `not_owned`,
as a way to reduce database reads on the primary DB. Check #462611.

To reduce resource saturation on the primary node, all workers should be changed to `sticky`, at minimum.

Since jobs are now enqueued along with the current database LSN, the replica (for `:sticky` or `:delayed`)
is guaranteed to be caught up to that point, or the job will be retried, or use the primary. Consider updating
the worker(s) to `delayed`, if it's applicable.

You can read more about the Sidekiq Workers `data_consistency` in
https://docs.gitlab.com/ee/development/sidekiq/worker_attributes.html#job-data-consistency-strategies.

You can use this [dashboard](https://log.gprd.gitlab.net/app/r/s/iyIUV) to monitor the worker query activity on
primary vs. replicas.

Currently, the `gitlab-housekeeper` is not always capable of updating all references, so you must check the diff
and pipeline failures to confirm if there are any issues.

This change was generated by
[gitlab-housekeeper](https://gitlab.com/gitlab-org/gitlab/-/tree/master/gems/gitlab-housekeeper)
using the Keeps::UpdateWorkersDataConsistency keep.

To provide feedback on your experience with `gitlab-housekeeper` please create an issue with the
label ~"GitLab Housekeeper" and consider pinging the author of this keep.

Changelog: other

# Conflicts:
#	.rubocop_todo/sidekiq_load_balancing/worker_data_consistency.yml
parent eb20c718
No related branches found
No related tags found
2 merge requests!171309Draft: Fetch issues / MRs / epics count in the sidebar with GraphQL,!170350Change data consistency for workers maintained by not_owned
......@@ -9,11 +9,6 @@ SidekiqLoadBalancing/WorkerDataConsistency:
- 'app/workers/bulk_imports/finish_batched_pipeline_worker.rb'
- 'app/workers/bulk_imports/pipeline_batch_worker.rb'
- 'app/workers/bulk_imports/relation_batch_export_worker.rb'
- 'app/workers/chaos/cpu_spin_worker.rb'
- 'app/workers/chaos/db_spin_worker.rb'
- 'app/workers/chaos/kill_worker.rb'
- 'app/workers/chaos/leak_mem_worker.rb'
- 'app/workers/chaos/sleep_worker.rb'
- 'app/workers/chat_notification_worker.rb'
- 'app/workers/ci/archive_traces_cron_worker.rb'
- 'app/workers/ci/build_prepare_worker.rb'
......
......@@ -4,7 +4,7 @@ module Chaos
class CpuSpinWorker # rubocop:disable Scalability/IdempotentWorker
include ApplicationWorker
data_consistency :always
data_consistency :sticky
sidekiq_options retry: 3
include ChaosQueue
......
......@@ -4,7 +4,7 @@ module Chaos
class DbSpinWorker # rubocop:disable Scalability/IdempotentWorker
include ApplicationWorker
data_consistency :always
data_consistency :sticky
sidekiq_options retry: 3
include ChaosQueue
......
......@@ -4,7 +4,7 @@ module Chaos
class KillWorker # rubocop:disable Scalability/IdempotentWorker
include ApplicationWorker
data_consistency :always
data_consistency :sticky
include ChaosQueue
sidekiq_options retry: false
......
......@@ -4,7 +4,7 @@ module Chaos
class LeakMemWorker # rubocop:disable Scalability/IdempotentWorker
include ApplicationWorker
data_consistency :always
data_consistency :sticky
sidekiq_options retry: 3
include ChaosQueue
......
......@@ -4,7 +4,7 @@ module Chaos
class SleepWorker # rubocop:disable Scalability/IdempotentWorker
include ApplicationWorker
data_consistency :always
data_consistency :sticky
sidekiq_options retry: 3
include ChaosQueue
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment