Update RepositoryUpdateMirrorWorker as having external_dependencies label
Summary
As part of the ongoing work to prepare the Sidekiq service to include it's SLA in our monthly reporting, the Reliability::General team is tempering the impact of external dependency jobs on the Sidekiq SLA.
Impact
During third party outages and maintenance windows, we see large spikes in failing Sidekiq jobs that are beyond the control of our platform. In an attempt to lessen the impact on our SLA from these, Sidekiq jobs labeled as having external_dependencies
were broken out into a separate monitoring component and their weight reduced.
Job failures by worker and external_dependency label during a third party outage: Source
Many(all?) of the GitLab::GithubImport
workers are labeled as such and were reduced in impact to our error rate. But RepositoryUpdateMirrorWorker
is not labeled as an external_dependency
. While the RepositoryUpdateMirrorWorker
job may find many uses as repository updates between GitLab.com and GitLab.com, it is also widely used for other GitLab instances (such as ops) and third party mirroring.
This issue has more specifics detailing creating and viewing this change in how we measure this type of work: https://gitlab.com/gitlab-com/gl-infra/reliability/-/issues/23793
Recommendation
Add the worker_has_external_dependencies
declaration to the RepositoryUpdateMirrorWorker
.
Verification
This is a thanos query showing the failed RepositoryUpdateMirrorWorker
jobs and it's external_dependencies
label:
Source