Skip to content

Concurent instances of bst should share cache without issues

Background

Currently we do not properly deal with concurrency in artifact cache.

For instance we could get the same issue #520 (closed), but while scanning removed artifacts.

Artifacts in cache should not be removed if still planned to be used by a task in a pipeline of any instance.

Also we should be able to add/remove artifacts/extracts concurrently without failure.

Acceptance Criteria

  • Scanning artifact/extract directories can handle deletion of artifacts/extracts.
  • Artifacts and extracts can be locked.
  • Adding and removing of artifacts is safe to do concurrently (I believe it is already)
  • Adding and removing of extracts is safe to do concurrently (I think it is not safe, we raise errors when we cannot commit extract directory).
  • Calculating/caching the size of artifact cache should work across instances

Edited by Valentin David
To upload designs, you'll need to enable LFS and have an admin enable hashed storage. More information