Skip to content

Upgrade fog-aws to v3.12.0

Stan Hu requested to merge sh-upgrade-fog-3.12.0 into master

What does this MR do?

This upgrade properly supports regional STS endpoints as well as the ability to set the minimum and maximum multipart chunk size.

How to setup and validate locally (strongly suggested)

Set up is a bit of a pain, but to test regional STS endpoints you need to somehow get an AWS Web Identity Token.

I got mine from setting up a K8S cluster with IAM Service Role (https://docs.aws.amazon.com/eks/latest/userguide/iam-roles-for-service-accounts-technical-overview.html). Then on some other machine, I saved AWS_WEB_IDENTITY_TOKEN_FILE TO a /tmp/identity.txt:

AWS_STS_REGIONAL_ENDPOINTS=regional AWS_WEB_IDENTITY_TOKEN_FILE=/tmp/identity.txt AWS_ROLE_ARN=arn:aws:iam::855262394183:REDACTED /opt/gitlab/embedded/bin/ruby -Ilib test-identity.rb

To test fog-aws directly (https://github.com/fog/fog-aws/), I used this script:

require 'fog-aws'

connection = Fog::Storage.new({provider: 'AWS', region: 'us-west-2', use_iam_profile: true})
dir = connection.directories.new(key: 'stanhu-s3-workhorse-testing')
data = "test" * 1024

f = dir.files.create(
  key: 'test.txt',
  body: data
)
puts f

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Edited by Stan Hu

Merge request reports