S3 cache not working with ECS Task IAM Role
Summary
We use GitLab Runner as described in Autoscaling GitLab Runner on AWS with the only exception that we run the runner itself not on a bastion host but as AWS ECS service on Fargate. Which has the nice property that we do not have to maintain a bastion EC2 VM. So far, everthing works quite well, the only thing which does not work is S3 cache. The reason for this is that GitLab Runner uses the minio-go client for S3 to create the presigned URL for up- respectively downloading the cache files to S3. minio-go does not (yet) support ECS Task Roles which causes the creation of the presigned URLs to fail as it cannot retrieve the AWS credentials. I created pull PR-1027 request for minio-go to add support for ECS Task Roles to minio-go. Once minio-go has support for ECS Task Roles the vendored minio-go dependency of GitLab Runner should be updated.
Manually applying the changes from PR-1027 to the vendored minio-go dependency I was able to build a GitLab Runner version which works fine (for us) on ECS and also supports S3 cache.
Steps to reproduce
Run GitLab Runner as ECS Service (Task) on AWS ECS with launch type FARGATE
using a Task Role with appropriate permissions for S3.
Actual behavior
GitLab Runner cannot create presigned URLs for uploading cache files to S3.
Expected behavior
GitLab Runner can crate presigned URLs for uploading cache files to S3.
Used GitLab Runner version
This affects at least the following GitLab Runner versions:
- 11.4.0
- 11.3.0
- 11.2.0