Gitlab Runner unable to access S3 Cache with IAM role
Summary
I get an Access Forbidden Error when trying to use an S3 cache.
Steps to reproduce
I'm using the helm chart to deploy.
config.yml
# PRODUCTION CLUSTER CONFIG
image: gitlab/gitlab-runner:alpine
imagePullPolicy: IfNotPresent
gitlabUrl: https://gitlab.<redacted>.com/ci
unregisterRunners: true
terminationGracePeriodSeconds: 3600
rbac:
clusterWideAccess: false
serviceAccountName: gitlab-runner
runners:
image: ubuntu:16.04
imagePullPolicy: always
tags: production-runner
runUntagged: true
privileged: false
secret: gitlab-runner
namespace: pipeline
cache:
cacheType: s3
cachePath: "production_runner_cache"
cacheShared: true
## S3 settings
s3ServerAddress: s3.amazonaws.com
s3BucketName: <redacted>-gitlab-runner
s3BucketLocation: us-west-2
s3CacheInsecure: false
podAnnotations:
cluster-autoscaler.kubernetes.io/safe-to-evict: true
iam.amazonaws.com/role: "/pipeline/gitlab-runner-role"
resources:
limits:
memory: 512Mi
cpu: 200m
requests:
memory: 256Mi
cpu: 100m
podAnnotations:
cluster-autoscaler.kubernetes.io/safe-to-evict: true
iam.amazonaws.com/role: "/pipeline/gitlab-runner-role".gitlab-ci.yml
image: golang:1.14-alpine
variables:
GOPATH: "${CI_PROJECT_DIR}/vendor/go"
cache:
key: ${CI_JOB_NAME}
paths:
- vendor/go/pkg/mod/
before_script:
- apk add git make gcc musl-dev bash
- go get github.com/jstemmer/go-junit-report
- go get -u golang.org/x/lint/golint
# We need to adjust path to account for cache
- export PATH="${GOPATH}/bin:${PATH}"
stages:
- lint
- test
...IAM Role
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "1",
"Effect": "Allow",
"Action": "s3:*",
"Resource": [
"arn:aws:s3:::<redacted>-gitlab-runner/*",
"arn:aws:s3:::<redacted>-gitlab-runner"
]
}
]
}Pod
$ kubectl describe pod runner-viu1yeku-project-496-concurrent-1qq8rf -n pipeline
Name: runner-viu1yeku-project-496-concurrent-1qq8rf
Namespace: pipeline
Priority: 0
Node: <redacted>
Start Time: Wed, 03 Jun 2020 14:32:30 -0400
Labels: pod=runner-viu1yeku-project-496-concurrent-1
Annotations: cluster-autoscaler.kubernetes.io/safe-to-evict: true
iam.amazonaws.com/role: /pipeline/gitlab-runner-roleActual behavior
Running after_script
00:00
Saving cache
00:01
Creating cache default...
WARNING: .go/pkg/mod/: no matching files
Uploading cache.zip to https://<redacted>-gitlab-runner.s3.dualstack.us-west-2.amazonaws.com/production_runner_cache/project/496/default
FATAL: received: 403 Forbidden
Failed to create cacheExpected behavior
Expected cache to work.
Used GitLab Runner version
Helm chart version - 0.12.0