Parallelize artifact deletion

@fabiopitino

Have you thought of breaking the list of files into batches and run a worker for each batch? From a Redis perspective the usage is pretty much the same but we can (1) parallelize the deletion and (2) having a shorter list it would ensure Sidekiq job doesn’t timeout and restart.

artifact_files.each_slice(DELETE_BATCH_SIZE) do |files|
  Ci::DeleteStoredArtifactsWorker.perform_async(files)
end
Assignee Loading
Time tracking Loading