Skip to content

Add 'imports' sidekiq shard

John Skarbek requested to merge jts/new-shard into master
  • New shard will take only one worker repository_import
    • This currently runs on the catchall shard
    • It'll be removed in a future commit
  • Placing this shard on the existing catchall shard as we appear to have plenty of available RAM on those nodes
  • Limiting concurrency to 1 to limit the blast radius when Pods are rotated for any reason
  • Locking scale to prevent auto-scale from unnecessarily removing in process jobs
  • Using the same CPU resource requests and limits as our catchall shard
    • As this workload is not considered urgent these values shall suffice
  • Doubling the RAM resources and requests due to seeing occasional OOM events on the catchall shard
  • It is estimated, that over a 7 day period of time, we should see a job saturation of 41%
  • For staging, we lower the memory requirements and the amount of replicas, because staging is slightly smaller

Addresses: gitlab-org/gitlab#332616 (closed)

What does this MR do?

Author Check-list

Please read the Contributing document and once you do, complete the following:

  • Assign to the correct reviewer per the contributing document
  • Apply the correct metadata per the contributing document

Reviewer Check-list

  • Reviewed the diff jobs to confirm changes are as expected
  • No changes shown in the diffs not associated with this MR - This may require a rebase or further investigation
Edited by John Skarbek

Merge request reports