GitLab Runner v17.5.2 may not work properly with AWS S3 if ServerAddress is configured

ZD: https://gitlab.zendesk.com/agent/tickets/577311

Suppose you have this config:

[runners.cache.s3]
ServerAddress = "s3.amazonaws.com"
BucketName = "example"
BucketLocation = "eu-south-1"
  1. With GitLab Runner v17.5.0, which uses the AWS SDK, the requests failed because ServerAddress points to the legacy global endpoint, https://s3.amazonaws.com.
  2. This previously worked in older GitLab Runner versions because the Minio SDK essentially ignores whatever is stored in ServerAddress as long as it ends with s3.amazonaws.com. For example, for a bucket in eu-south-1, the Minio SDK looks up the endpoint for eu-south-1 and returns s3.dualstack.eu-south-1.amazonaws.com.
  3. In contrast, the AWS SDK assumes that if you provide a custom ServerAddress then that it should use that.

!5111 (merged) fixes the issue if s3.amazonaws.com is used, but what happens if a user sets ServerAddress to one of the following?

  1. eu-south-1.s3.amazonaws.com
  2. example.eu-south-1.s3.amazonaws.com
  3. bogus-hostname.eu-south-1.s3.amazonaws.com

These values "magically" worked with the Minio SDK, but these values no longer work in GitLab Runner v17.5.0.

I'm not sure how big of a problem this really is at the moment. We document the use of s3.amazonaws.com in https://docs.gitlab.com/runner/configuration/advanced-configuration.html#the-runnerscaches3-section for AWS, so !5111 (merged) might be sufficient.

To make those config values above compatible, we would have to replicate the Minio logic in https://github.com/minio/minio-go/blob/13faa64c02170d2256e21446b63f3964fd219aaa/api.go#L929-L947 and also look for <bucket>.s3.amazonaws.com and <bucket>.<region>.s3.amazonaws.com. It's a little tricky because we also have to consider S3 accelerated endpoints.

Edited by Stan Hu