Skip to content

Do not count delete marked blobs to cleanup total

Cal Pratt requested to merge cpratt34/cleanup-include-exclude-marked into master

When we count blobs for deletion we always look to at the marked delete items first. When multiple cleanup daemons are running, they fight over who can delete the marked items first. Those already marked records cut into the total each job is attempting to clean up. This leads to less real work being performed when using multiple cleaners. The redundant work makes the cleaners not able to keep up with load.

With this change, after the first run of a batched cleanup, we stop counting the already delete marked items to our calculated counts, because in this state marked items should already be managed by another cleaner. This should lead to a better sample of digests being selected on each batch.

Merge request reports