Concurent instances of bst should share cache without issues
Background
Currently we do not properly deal with concurrency in artifact cache.
For instance we could get the same issue #520 (closed), but while scanning removed artifacts.
Artifacts in cache should not be removed if still planned to be used by a task in a pipeline of any instance.
Also we should be able to add/remove artifacts/extracts concurrently without failure.
Acceptance Criteria
- Scanning artifact/extract directories can handle deletion of artifacts/extracts.
- Artifacts and extracts can be locked.
- Adding and removing of artifacts is safe to do concurrently (I believe it is already)
- Adding and removing of extracts is safe to do concurrently (I think it is not safe, we raise errors when we cannot commit extract directory).
- Calculating/caching the size of artifact cache should work across instances
Edited by Valentin David