Detect bucket location when not provided

What does this MR do?

Add bucket location detection for adpater v2.

Why was this MR needed?

To fix a regression which was introduced in the s3 adapter v2

What's the best way to test this MR?

  1. Connect to AWS and switch the default region to another one us-west-1 by example
  2. From this region, create a s3 bucket. touni-test-bucket-west by example
  3. Configure GitLab Runner with the config below
config.toml
listen_address = ":9252"
concurrent = 3
check_interval = 1
log_format = "runner"
connection_max_age = "15m0s"
shutdown_timeout = 0

[session_server]
  session_timeout = 1800

[[runners]]
  name = "investigation"
  limit = 50
  url = "https://gitlab.com/"
  id = 0
  token = "glrt-REDACTED"
  token_obtained_at = 2024-09-30T14:38:04.623237Z
  executor = "kubernetes"
  shell = "bash"
  [runners.cache]
    Type = "s3"
    Shared = false
  [runners.cache.s3]
    ServerAddress = "s3.amazonaws.com"
    AccessKey = "REDACTED"
    SecretKey = "REDACTED"
    BucketName = "touni-test-bucket-west"
  [runners.feature_flags]
    FF_USE_ADVANCED_POD_SPEC_CONFIGURATION = true
    FF_USE_POD_ACTIVE_DEADLINE_SECONDS = true
    FF_PRINT_POD_EVENTS = true
    FF_USE_FASTZIP = true
  [runners.kubernetes]
    host = ""
    bearer_token_overwrite_allowed = false
    image = ""
    namespace = ""
    namespace_overwrite_allowed = ""
    namespace_per_job = false
    privileged = true
    node_selector_overwrite_allowed = ".*"
    node_tolerations_overwrite_allowed = ""
    pod_labels_overwrite_allowed = ""
    service_account_overwrite_allowed = ""
    [runners.kubernetes.pod_labels]
    [[runners.kubernetes.volumes.empty_dir]]
      name = "docker-certs"
      mount_path = "/certs/client"
      medium = "Memory"
    [runners.kubernetes.dns_config]
  1. Use the gitlab-ci yaml below
gitlab-ci.yaml
variables:
  FF_USE_ADVANCED_POD_SPEC_CONFIGURATION: "true"
  FF_USE_POWERSHELL_PATH_RESOLVER: "true"
  FF_RETRIEVE_POD_WARNING_EVENTS: "true"
  FF_PRINT_POD_EVENTS: "true"

job:
  image: alpine
  cache:
    paths:
      - cache_file
  script:
    - sleep 10
    - mkdir -p cache_file
    - touch cache_file/test.txt
  1. The job should pass without any issue. In the job log, the region should be the bucket location us-west-1 for our example
  2. When the job is restarted the bucket location is still correctly detected

What are the relevant issue numbers?

close #38345 (closed)

Merge request reports

Loading