Skip to content

Add awscli as an optional s3 interface tool

Jeremy Huntwork requested to merge jhuntwork/CNG:master into master

What does this MR do?

Adds the aws CLI as an optional tool for interfacing with the S3 backend.

Main motivations are:

  • This tool is developed by AWS and is their standard for interacting with their API from the command line
  • Better credential handling, including support for IMDSv2. This means that a container running on an EC2 instance can automatically use IAM roles and permissions associated with the instance. And when running in Kubernetes (like in EKS) a native OIDC implementation will allow a pod to receive a specific role when using a service account annotated with the right role. This 'just works' without having to maintain a specific set of credentials. See: https://aws.amazon.com/blogs/opensource/introducing-fine-grained-iam-roles-service-accounts/ Lastly, credentials could just be passed in via environment variables, which also 'just works' with the CLI.

Related issues

gitlab-org/gitlab#334160

Checklist

See Definition of done.

For anything in this list which will not be completed, please provide a reason in the MR discussion

Required

  • Merge Request Title, and Description are up to date, accurate, and descriptive
  • MR targeting the appropriate branch
  • MR has a green pipeline on GitLab.com

Expected (please provide an explanation if not completing)

  • Test plan indicating conditions for success has been posted and passes
  • Documentation created/updated
  • Integration tests added to GitLab QA
  • The impact any change in container size has should be evaluated

Uncertain how you would like this to be tested, but have done partial testing on a personal EKS cluster. Happy to test in additional ways.

Edited by Jeremy Huntwork

Merge request reports