Jira ImportIssueWorker should not write unused key into redis-cache
Background
There are ongoing discussions on handling application logic changes to prepare for zonal sidekiq (gitlab-com/gl-infra/scalability#973). Only 1 sidekiq worker contains application-side limiting logic which checks sidekiq queue length to decide if a pause needs to be triggered.
The context for this pause-logic can be found in (high volume of cache:gitlab:jira-import*
keys found) gitlab-com/gl-infra/scalability#419 (comment 381941081) and (proposal to add application limits for jira imports) #244859 (closed).
Issue
The question to answer is: Could ImportIssueWorker
stop writing the items-mapper key into redis-cache?
The cache key is cache:gitlab:jira-import/items-mapper/NUM/issues/NUM
(written by https://gitlab.com/gitlab-org/gitlab/-/blob/master/app/workers/gitlab/jira_import/import_issue_worker.rb#L19).
The cache key cache:gitlab:jira-import/items-mapper/NUM/issues/NUM
in redis-cache contributes to a large volume of keys when large jira projects are imported. However, these keys are not used anywhere else in the application (noted in MR discussion: !27229 (comment 311307225) and verified with @acroitor) nor is it cleaned up immediately after a finished import (its removed via 24h TTL).
Application limits were imposed by using sidekiq queue length as a measure (!93577 (merged)) to prevent such spikes. However, such spikes can be avoided altogether if this key is removed.
Doing so will benefit zonal sidekiq efforts in getting application logic ready for zonal sidekiq adoption. We can safely use an alternative measure to perform application-side limiting that does not depend on sidekiq queue length.