Triage non-idempotent workers

At gitlab-org/gitlab!25202 (merged) we're introducing a idempotent cop check for the new idempotent worker attribute. It bypasses a series of workers that could be idempotent, but both need further triage and testing in order to receive a idempotent! call.

The workers that schedule the most duplicates (duplicates are jobs for the same worker, with the same arguments, while another job is already enqueued).

json.class.keyword: Descending Count Sum of json.duration Duration in hours Issue
AuthorizedProjectsWorker 1,169,309 1,631,372.94 453.1591506 #280 (closed)
ProcessCommitWorker 1,028,703 434,819.12 120.7830881 #282 (closed)
WebHookWorker 1,537,784 260,819.55 72.44987528 gitlab-org/gitlab#218559 (closed)
ExpirePipelineCacheWorker 439,763 88,945.36 24.70704417 #284 (closed)
StageUpdateWorker 747,685 87,157.24 24.21034417 #285 (closed)
ProjectDailyStatisticsWorker 479,511 70,250.28 19.51396556 #256 (closed)
PipelineUpdateWorker 308,843 32,321.07 8.978074444 #287 (moved)
Namespaces::ScheduleAggregationWorker 186,661 16,433.43 4.564840278 #218 (closed)
UpdateHeadPipelineForMergeRequestWorker 238,564 13,914.08 3.8650225 #288 (closed)
ElasticIndexerWorker 102,903 11,922.26 3.311739444 #289 (closed)
ExpireJobCacheWorker 57,341 11,189.24 3.108121667 #290 (closed)
BuildHooksWorker 27,700 11,183.11 3.106418611 #291 (moved)
ProjectImportScheduleWorker 16,119 8,245.43 2.290396944
UpdateAllMirrorsWorker 2,526 6,537.76 1.816045
CleanupContainerRepositoryWorker 446 5,442.65 1.5118475
RepositoryUpdateRemoteMirrorWorker 231 5,171.90 1.436639722
ElasticCommitIndexerWorker 11,387 4,259.01 1.1830575
ProjectCacheWorker 83,969 4,166.36 1.157321667
MergeRequestMergeabilityCheckWorker 2,674 4,124.13 1.145591944
ProjectServiceWorker 71,781 3,944.42 1.095673333
ElasticIndexBulkCronWorker 2,260 1,068.40 0.2967786111
AutoMergeProcessWorker 4,179 857.239 0.2381219444
RepositoryUpdateMirrorWorker 638 686.928 0.1908133333
ReactiveCachingWorker 310 454.489 0.1262469444
PipelineHooksWorker 2,215 415.77 0.1154916667

Source: https://log.gprd.gitlab.net/goto/a7fc2366e28f6321294d027ee376d92d

Edited by Luke Duncalfe