ETag mismatch for external storage when using e.g. Alibaba Cloud
Summary
When using the external object storage functionality as described in [https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer] it is not possible to upload to Alibaba Cloud (Aliyun) OSS due to a mismatch in how the ETag is calculated (Aliyun uses upper-case and AWS uses lower-case [https://www.alibabacloud.com/help/doc-detail/64919.htm]).
Steps to reproduce
- Setup deployed Gitlab Helm Chart to use external storage buckets at Aliyun OSS
- Try to upload an artifact in your .gitlab-ci.yml
Configuration used
rails.yaml used to setup connection
# Example configuration of `connection` secret for Rails
# Example for AWS S3
# See https://gitlab.com/charts/gitlab/blob/master/doc/charts/globals.md#connection
# See https://gitlab.com/charts/gitlab/blob/master/doc/advanced/external-object-storage
provider: AWS
# Specify the region
region: eu-central-1
# Specify access/secret keys
aws_access_key_id: <ACCESS_KEY>
aws_secret_access_key: <ACCESS_SECRET>
# The below settings are for S3 compatible endpoints
# See https://docs.gitlab.com/ee/administration/job_artifacts.html#s3-compatible-connection-settings
host: https://oss-eu-central-1.aliyuncs.com
# aws_signature_version: 4
enable_signature_v4_streaming: false
# host: storage.example.com
# endpoint: "https://minio.example.com:9000"
path_style: true
.gitlab-ci.yml
# Modified based on: https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/
stages:
- build_sdist
Build sdist:
stage: build_sdist
image: python:3.7
artifacts:
paths:
- dist/$CI_PROJECT_NAME-*.tar.gz
expire_in: 1 week
script:
- make build
Current behavior
Output inside gitlab-workhorse:
time="2019-09-02T12:44:33Z" level=error msg=error correlation_id=E4vfraV1Ho4 error="handleFileUploads: extract files from multipart: persisting multipart file: upload part 1: ETag mismatch. expected \"018b158a0c6853c99eae7205bb70db11\" got \"018B158A0C6853C99EAE7205BB70DB11\"" method=POST uri="/api/v4/jobs/101/artifacts?artifact_format=zip&artifact_type=archive&expire_in=1+week"
Expected behavior
The artifact to upload correctly.
Versions
- Chart: gitlab-2.2.1
- Platform:
- Cloud: Alibaba Cloud
- Kubernetes: (
kubectl version
)- Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
- Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.6-aliyun.1", GitCommit:"8cb561c", GitTreeState:"", BuildDate:"2019-04-22T11:34:20Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
- Helm: (
helm version
)- Client: &version.Version{SemVer:"v2.10.0", GitCommit:"9ad53aac42165a5fadc6c87be0dea6b115f93090", GitTreeState:"clean"}
- Server: &version.Version{SemVer:"v2.14.1", GitCommit:"5270352a09c7e8b6e8c9593002a73535276507c0", GitTreeState:"clean"}
Relevant logs
(Please provide any relevate log snippets you have collected, using code blocks (```) to format)