Potential: Update ExpireJobCacheWorker to utilize new Sidekiq readonly database replicas capabilities
With the ability to Allow Sidekiq jobs to use readonly database replicas in place, the next step will be to enable this capability for this worker.
For more context, please check the following links,
- Slides: https://slides.com/nikolamilojevic/deck-eb7ff8/fullscreen
- A recorded Show and Tell introducing Sidekiq Load Balancing, https://youtu.be/PAJ8nvG1AGA
- Documentation(WIP): !59397 (merged)
- An example of shifting a Sidekiq worker to database replica: #324377 (closed)
Proposal
Use data_consistency :sticky
because it's the best trade-off. A data consistency delayed
could cause the job to be rescheduled if the replica hasn't caught up, but the worker is set to run with urgency :high
so we want to avoid that. A sticky
mode still allows us to use replica as much as possible.
Designs
- Show closed items
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
- Kamil Trzciński added to epic &5592 (closed)
added to epic &5592 (closed)
- Darby Frey added rapid actionshift db load from primary label
added rapid actionshift db load from primary label
- Darby Frey marked this issue as related to #322452 (closed)
marked this issue as related to #322452 (closed)
- Darby Frey added workflowblocked label
added workflowblocked label
- Nikola Milojevic changed epic to &5691 (closed)
changed epic to &5691 (closed)
- Kamil Trzciński mentioned in merge request !57773 (merged)
mentioned in merge request !57773 (merged)
- Changzheng Liu marked #324379 (closed) as a duplicate of this issue
marked #324379 (closed) as a duplicate of this issue
- Changzheng Liu marked this issue as related to #324379 (closed)
marked this issue as related to #324379 (closed)
- Changzheng Liu added grouppipeline execution label
added grouppipeline execution label
- Developer
@cheryl.li assign this issue to you as I understand it's owned by ~"group::continuous integration"
- Changzheng Liu added workflowready for development label and removed workflowblocked label
added workflowready for development label and removed workflowblocked label
- Changzheng Liu assigned to @cheryl.li
assigned to @cheryl.li
- Maintainer
@jreporter Another infradev follow-up from the rapid actionshift db load from primary (goal 3), now that the issue it's being blocked by is now fully deployed in Production. The work should be pretty straightforward as we would follow what we did to migrate the
BuildHooksWorker
: #324232 (closed) Collapse replies - Developer
@cheryl.li - I have triaged this for the 14.3 needs weight and scheduled for 14.4
- Developer
@jreporter @cheryl.li , as suggested in the daily standup, would you mind also setting a due date? Thanks.
- Developer
Sure @changzhengliu - I will align it with the milestone currently assigned!
1
- Cheryl Li added backend infradev severity2 workflowplanning breakdown labels and removed rapid actionshift db load from primary workflowready for development labels
added backend infradev severity2 workflowplanning breakdown labels and removed rapid actionshift db load from primary workflowready for development labels
- Cheryl Li removed the relation with #322452 (closed)
removed the relation with #322452 (closed)
- Cheryl Li marked this issue as related to #322452 (closed)
marked this issue as related to #322452 (closed)
- Cheryl Li unassigned @cheryl.li
unassigned @cheryl.li
- Maintainer
All infradev issues need to have a proper severity label, priority label label, and a milestone set. Please add those to this issue. For more details, see the handbook.
- 🤖 GitLab Bot 🤖 added automation:infradev-missing-labels label
added automation:infradev-missing-labels label
- Maintainer
Setting label(s) devopsverify sectionops based on ~"group::continuous integration".
- 🤖 GitLab Bot 🤖 added devopsverify sectionops labels
added devopsverify sectionops labels
- Changzheng Liu changed epic to &5983 (closed)
changed epic to &5983 (closed)
- Changzheng Liu mentioned in epic &5983 (closed)
mentioned in epic &5983 (closed)
- Jackie Porter added needs weight label
added needs weight label
- Jackie Porter changed milestone to %14.4
changed milestone to %14.4
- Jackie Porter mentioned in issue #330664 (closed)
mentioned in issue #330664 (closed)
- Jackie Porter changed due date to October 17, 2021
changed due date to October 17, 2021
- Cheryl Li added Verify candidate label
added Verify candidate label
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3131 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3131 (closed)
- Jackie Porter added priority2 label
added priority2 label
- Changzheng Liu changed the description
Compare with previous version changed the description
- 🤖 GitLab Bot 🤖 removed automation:infradev-missing-labels label
removed automation:infradev-missing-labels label
- Fabio Pitino changed the description
Compare with previous version changed the description
- Maintainer
@jreporter @cheryl.li This change is trivial but setting
weight: 2
because we would want to roll this out with feature flag so there is that extra overhead. 1 Collapse replies - Developer
Okay, got it thanks for the weight and explanation @fabiopitino
- Fabio Pitino added workflowready for development label and removed needs weight workflowplanning breakdown labels
added workflowready for development label and removed needs weight workflowplanning breakdown labels
- Fabio Pitino set weight to 2
set weight to 2
- 🤖 GitLab Bot 🤖 added [deprecated] Accepting merge requests label
added [deprecated] Accepting merge requests label
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3222 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3222 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3226 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3226 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3294 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3294 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3298 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3298 (closed)
- Jackie Porter added Category:Continuous Integration label
added Category:Continuous Integration label
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3389 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3389 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3490 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3490 (closed)
- Matthias Käppler mentioned in merge request !62073 (merged)
mentioned in merge request !62073 (merged)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3564 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3564 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3662 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3662 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3751 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3751 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3848 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3848 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#3913 (closed)
mentioned in issue gitlab-org/quality/triage-reports#3913 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#4005 (closed)
mentioned in issue gitlab-org/quality/triage-reports#4005 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#4014 (closed)
mentioned in issue gitlab-org/quality/triage-reports#4014 (closed)
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#4099 (closed)
mentioned in issue gitlab-org/quality/triage-reports#4099 (closed)
- Jackie Porter mentioned in issue gitlab-org/ci-cd/pipeline-execution#70 (closed)
mentioned in issue gitlab-org/ci-cd/pipeline-execution#70 (closed)
- Developer
@jreporter grouppipeline authoring will take this issue for %14.3
Collapse replies - Developer
Wonderful thank you @dhershkovitch
- Dov Hershkovitch changed milestone to %14.3
changed milestone to %14.3
- Dov Hershkovitch added grouppipeline authoring + 1 deleted label and removed grouppipeline execution label
added grouppipeline authoring + 1 deleted label and removed grouppipeline execution label
- Dov Hershkovitch mentioned in issue gitlab-org/ci-cd/pipeline-authoring#25 (closed)
mentioned in issue gitlab-org/ci-cd/pipeline-authoring#25 (closed)
- Laura Montemayor assigned to @lauraX
assigned to @lauraX
- Laura Montemayor unassigned @lauraX
unassigned @lauraX
- Developer
Reassigning to @nagyv-gitlab’s group
1 - Jackie Porter added groupconfigure [DEPRECATED] label and removed grouppipeline authoring label
added groupconfigure [DEPRECATED] label and removed grouppipeline authoring label
- Viktor Nagy (GitLab) added candidate14.3 label
added candidate14.3 label
- Cheryl Li mentioned in issue #296616 (closed)
mentioned in issue #296616 (closed)
- Jackie Porter removed 1 deleted label
removed 1 deleted label
- Jackie Porter removed candidate14.3 label
removed candidate14.3 label
- Jackie Porter added grouppipeline execution label and removed groupconfigure [DEPRECATED] label
added grouppipeline execution label and removed groupconfigure [DEPRECATED] label
- Developer
Updated the labels @cheryl.li
1 - Jackie Porter added All SaaS FY22Q3 reliability labels
added All SaaS FY22Q3 reliability labels
- 🤖 GitLab Bot 🤖 mentioned in issue gitlab-org/quality/triage-reports#4264 (closed)
mentioned in issue gitlab-org/quality/triage-reports#4264 (closed)
- Laura Montemayor assigned to @lauraX
assigned to @lauraX
- 🤖 GitLab Bot 🤖 removed [deprecated] Accepting merge requests label
removed [deprecated] Accepting merge requests label
- Cheryl Li added Engineering Allocation label
added Engineering Allocation label