You need to sign in or sign up before continuing.
ETag mismatch for external storage when using e.g. Alibaba Cloud
Summary
When using the external object storage functionality as described in [https://docs.gitlab.com/charts/advanced/external-object-storage/#lfs-artifacts-uploads-packages-external-diffs-pseudonymizer] it is not possible to upload to Alibaba Cloud (Aliyun) OSS due to a mismatch in how the ETag is calculated (Aliyun uses upper-case and AWS uses lower-case [https://www.alibabacloud.com/help/doc-detail/64919.htm]).
Steps to reproduce
- Setup deployed Gitlab Helm Chart to use external storage buckets at Aliyun OSS
- Try to upload an artifact in your .gitlab-ci.yml
Configuration used
rails.yaml used to setup connection
# Example configuration of `connection` secret for Rails
# Example for AWS S3
# See https://gitlab.com/charts/gitlab/blob/master/doc/charts/globals.md#connection
# See https://gitlab.com/charts/gitlab/blob/master/doc/advanced/external-object-storage
provider: AWS
# Specify the region
region: eu-central-1
# Specify access/secret keys
aws_access_key_id: <ACCESS_KEY>
aws_secret_access_key: <ACCESS_SECRET>
# The below settings are for S3 compatible endpoints
# See https://docs.gitlab.com/ee/administration/job_artifacts.html#s3-compatible-connection-settings
host: https://oss-eu-central-1.aliyuncs.com
# aws_signature_version: 4
enable_signature_v4_streaming: false
# host: storage.example.com
# endpoint: "https://minio.example.com:9000"
path_style: true
.gitlab-ci.yml
# Modified based on: https://blog.callr.tech/building-docker-images-with-gitlab-ci-best-practices/
stages:
- build_sdist
Build sdist:
stage: build_sdist
image: python:3.7
artifacts:
paths:
- dist/$CI_PROJECT_NAME-*.tar.gz
expire_in: 1 week
script:
- make build
Current behavior
Output inside gitlab-workhorse:
time="2019-09-02T12:44:33Z" level=error msg=error correlation_id=E4vfraV1Ho4 error="handleFileUploads: extract files from multipart: persisting multipart file: upload part 1: ETag mismatch. expected \"018b158a0c6853c99eae7205bb70db11\" got \"018B158A0C6853C99EAE7205BB70DB11\"" method=POST uri="/api/v4/jobs/101/artifacts?artifact_format=zip&artifact_type=archive&expire_in=1+week"
Expected behavior
The artifact to upload correctly.
Versions
- Chart: gitlab-2.2.1
- Platform:
- Cloud: Alibaba Cloud
- Kubernetes: (
kubectl version
)- Client Version: version.Info{Major:"1", Minor:"15", GitVersion:"v1.15.3", GitCommit:"2d3c76f9091b6bec110a5e63777c332469e0cba2", GitTreeState:"clean", BuildDate:"2019-08-19T11:13:54Z", GoVersion:"go1.12.9", Compiler:"gc", Platform:"linux/amd64"}
- Server Version: version.Info{Major:"1", Minor:"12+", GitVersion:"v1.12.6-aliyun.1", GitCommit:"8cb561c", GitTreeState:"", BuildDate:"2019-04-22T11:34:20Z", GoVersion:"go1.10.8", Compiler:"gc", Platform:"linux/amd64"}
- Helm: (
helm version
)- Client: &version.Version{SemVer:"v2.10.0", GitCommit:"9ad53aac42165a5fadc6c87be0dea6b115f93090", GitTreeState:"clean"}
- Server: &version.Version{SemVer:"v2.14.1", GitCommit:"5270352a09c7e8b6e8c9593002a73535276507c0", GitTreeState:"clean"}
Relevant logs
(Please provide any relevate log snippets you have collected, using code blocks (```) to format)