Skip to content

aws: Adding server side encryption configuration for s3 buckets

Matt Veitas requested to merge eb-veitas/gitlab-environment-toolkit:s3-encryption into main

What does this MR do?

Enables S3 encryption configuration for S3 buckets create by GET. This is a backwards compatible change as resources are modified in place and no resources are destroyed. The code utilizes the dynamic block to allow the server_side_encryption_configuration to be fully specified as a variable.

If this change is applied to existing buckets, only objects that are created afterwards would be encrypted: https://aws.amazon.com/premiumsupport/knowledge-center/s3-aws-kms-default-encryption/

TF example code

# Configure the AWS Provider
provider "aws" {
  region = "us-east-2"
}

module "gitlab_ref_arch_aws" {
  source = "../gitlab_ref_arch_aws"

  create_network = true

  prefix = "evbg-s3"

  ssh_public_key_file = "xxxxxxxxxx"
}

TF Plan

# module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["artifacts"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-artifacts"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["backups"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-backups"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["dependency-proxy"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-dependency-proxy"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["lfs"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-lfs"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["mr-diffs"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-mr-diffs"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["packages"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-packages"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["registry"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-registry"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["terraform-state"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-terraform-state"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["uploads"] will be created
  + resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
      + acceleration_status         = (known after apply)
      + acl                         = "private"
      + arn                         = (known after apply)
      + bucket                      = "evbg-s3-uploads"
      + bucket_domain_name          = (known after apply)
      + bucket_regional_domain_name = (known after apply)
      + force_destroy               = true
      + hosted_zone_id              = (known after apply)
      + id                          = (known after apply)
      + region                      = (known after apply)
      + request_payer               = (known after apply)
      + tags_all                    = (known after apply)
      + website_domain              = (known after apply)
      + website_endpoint            = (known after apply)

      + versioning {
          + enabled    = (known after apply)
          + mfa_delete = (known after apply)
        }
    }

Add server side encryption to S3 buckets

module "gitlab_ref_arch_aws" {
  source = "../gitlab_ref_arch_aws"

  create_network = true

  prefix = "evbg-s3"

  ssh_public_key_file = "xxxxxxxxx"

  object_storage_server_side_encryption_configuration = {
    rule = {
      apply_server_side_encryption_by_default = {
        sse_algorithm = "AES256"
      }
    }
  }

TF Plan

Terraform used the selected providers to generate the following execution plan. Resource actions are indicated with the following symbols:
  ~ update in-place

Terraform will perform the following actions:

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["artifacts"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-artifacts"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["backups"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-backups"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["dependency-proxy"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-dependency-proxy"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["lfs"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-lfs"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["mr-diffs"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-mr-diffs"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["packages"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-packages"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["registry"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-registry"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["terraform-state"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-terraform-state"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

  # module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["uploads"] will be updated in-place
  ~ resource "aws_s3_bucket" "gitlab_object_storage_buckets" {
        id                          = "evbg-s3-uploads"
        tags                        = {}
        # (10 unchanged attributes hidden)

      + server_side_encryption_configuration {
          + rule {
              + apply_server_side_encryption_by_default {
                  + sse_algorithm = "AES256"
                }
            }
        }

        # (1 unchanged block hidden)
    }

Plan: 0 to add, 9 to change, 0 to destroy.

TF Apply

module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["registry"]: Modifying... [id=evbg-s3-registry]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["uploads"]: Modifying... [id=evbg-s3-uploads]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["mr-diffs"]: Modifying... [id=evbg-s3-mr-diffs]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["artifacts"]: Modifying... [id=evbg-s3-artifacts]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["lfs"]: Modifying... [id=evbg-s3-lfs]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["backups"]: Modifying... [id=evbg-s3-backups]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["packages"]: Modifying... [id=evbg-s3-packages]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["dependency-proxy"]: Modifying... [id=evbg-s3-dependency-proxy]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["terraform-state"]: Modifying... [id=evbg-s3-terraform-state]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["dependency-proxy"]: Modifications complete after 2s [id=evbg-s3-dependency-proxy]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["registry"]: Modifications complete after 2s [id=evbg-s3-registry]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["backups"]: Modifications complete after 2s [id=evbg-s3-backups]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["terraform-state"]: Modifications complete after 2s [id=evbg-s3-terraform-state]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["packages"]: Modifications complete after 2s [id=evbg-s3-packages]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["artifacts"]: Modifications complete after 3s [id=evbg-s3-artifacts]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["mr-diffs"]: Modifications complete after 3s [id=evbg-s3-mr-diffs]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["lfs"]: Modifications complete after 3s [id=evbg-s3-lfs]
module.gitlab_ref_arch_aws.aws_s3_bucket.gitlab_object_storage_buckets["uploads"]: Modifications complete after 3s [id=evbg-s3-uploads]

Related issues

https://gitlab.com/gitlab-org/quality/gitlab-environment-toolkit/-/issues/272

Author's checklist

When ready for review, the Author applies the workflowready for review label and mention @gl-quality/get-maintainers:

  • Merge request:
    • Corresponding Issue raised and reviewed by the GET maintainers team.
    • Merge Request Title and Description are up to date, accurate, and descriptive
    • MR targeting the appropriate branch
    • MR has a green pipeline
  • Code:
    • Check the area changed works as expected. Consider testing it in different environment sizes (1k,3k,10k,etc.).
    • Documentation created/updated in the same MR.
    • If this MR adds an optional configuration - check that all permutations continue to work.
    • For Terraform changes: setup a previous version environment, then run a terraform plan with your new changes and ensure nothing will be destroyed. If anything will be destroyed and this can't be avoided please add a comment to the current MR.
  • Create any follow-up issue(s) to support the new feature across other supported cloud providers or advanced configurations. Create 1 issue for each provider/configuration. Contact the Quality Enablement team if unsure.
Edited by Matt Veitas

Merge request reports