Resolve "Optimize the garbage collector sweep stage for S3"

Context

Please see gitlab#31071 (closed) and #10 (closed) for context.

Results

The changes proposed here have shown encouraging results. Optimizations were performed in two main parts:

• Blob deletion
• Manifest deletion

Each of these MRs has a detailed analysis of the current behaviour and benchmarks of the proposed improvements. Please look at them for more information.

Tasks

• Avoid redundant digest cast and parse operations (!29 (merged))
• Add blob deletion benchmarks (!41 (merged) and !42 (merged))
• Implement bulk deletion for blobs (!31 (merged))
• Implement concurrency for bulk deletion (!32 (merged))
• Add manifest deletion benchmarks (!33 (merged))
• Add error handling tests for the new S3 driver DeleteFiles method (!38 (merged))
• Implement bulk deletion for manifests (!39 (merged))
• Remove empty folders left behind with the local filesystem storage driver (!47 (merged))
• Avoid stat calls for each tag being deleted (!52 (merged))
• Improve logging (!50 (merged))
• Detailed performance and cost analysis