Support sharding the S3 ActionCache into multiple buckets

Adam Coldrickrequested to merge
sotk/action-cache/s3-bucket-sharding into master

Before raising this MR, consider whether the following are required, and complete if so:

  • Unit tests
  • Metrics
  • Documentation update(s)

If not required, please explain in brief why not.

Description

This MR adds support for template bucket names in the S3 ActionCache configuration. The CAS S3 backend supports this already, so its a bit inconsistent that the ActionCache doesn't also support it. The lack of support for this means the maximum number of cached results is the maximum bucket size, which is a needless artificial limitation. Supporting templates to allow sharding based on the Action Digest greatly reduces the impact of this restriction.

Validation

Deploy a BuildGrid with an S3 ActionCache with the bucket set to something like cache-bucket--{digest[0]}{digest[1]} and make sure the cache works as expected.

Edited by Adam Coldrick

Merge request reports