Skip to content

Add support for server side encryption for S3 Cache

Johan Lanzrein requested to merge lanzrein/gitlab-runner:cache-s3-encryption into main

What does this MR do?

  • Update of minio-go to version 7.0.16 (was 7.0.13) to have access to the method PresignHeader. (Not needed as Gitlab is now using v7.0.24-0.20220302185830-f2d40be3e82f)

  • Modify the adapter for S3 cache. It replaces PresignPutObject and PresignGetObject with the PresignHeader method.

  • With PresignHeader, we can pass headers in the presigned URLs. This allows the cache to support server side encryption for S3.

  • This MR also makes changes in the configuration of the S3 cache configuration. The S3 Cache configuration supports two new attributes ServerSideEncryption and ServerSideEncryptionKeyId. These attributes are used to implement server side encryption.

  • The MR updates the method GetUploadHeaders for the S3 adapter. This method now returns headers based on the S3 configuration and supports server side encryption.

Why was this MR needed?

Without the capacity to add the headers, it is not possible to implement server side encryption for S3.

This MR adds support for Server Side encryption in the use of the S3 Cache. More specifically it adds support for the scenario where uploads are forbidden if encryption is not specified in the request. This scenario happens in setting where we want to avoid uploads of any unencrypted objects to S3. (See here for more information).

What's the best way to test this MR?

A concrete example would be to configure a runner with the following s3.cache config section:

[runners.cache]
  Type = "s3"
  Path = "path/to/prefix"
  Shared = false
  [runners.cache.s3]
    ServerAddress = "s3.amazonaws.com"
    AccessKey = "AWS_S3_ACCESS_KEY"
    SecretKey = "AWS_S3_SECRET_KEY"
    BucketName = "runners-cache"
    BucketLocation = "eu-west-1"
    Insecure = false
    ServerSideEncryption = "sse-kms"
    ServerSideEncryptionKeyId = "alias/my-key"

Then use the runner with the cache against a bucket that enforces KMS encryption with an IAM Policy like the one mentionned in this blogpost.

What are the relevant issue numbers?

Closes #3232 (closed)

Closes #28200 (closed)

Edited by Romuald Atchadé

Merge request reports