Skip to content

Fix Azure cache uploads using Go Cloud

Stan Hu requested to merge sh-add-gocloud-support-cache-archiver into master

Using a pre-signed PUT URL for uploading Azure blobs only works for objects up to 4 MB. Azure complains with "413 The request body is too large and exceeds the maximum permissible limit" for blobs that exceed this limit.

To fix this, we add an Azure client directly into the cache-archiver subcommand using Go Cloud and pass in credentials via environment variables. Go Cloud allows us to support many different object storage providers with little effort.

Since environment variables can be exposed with debug logging enabled, we switch from using an Account SAS (https://docs.microsoft.com/en-us/rest/api/storageservices/create-account-sas) to a Service SAS (https://docs.microsoft.com/en-us/rest/api/storageservices/create-service-sas). Like a pre-signed URL, this token is only valid for one specific blob for a limited time:

image

Closes #27123 (closed)

Edited by Stan Hu

Merge request reports