Skip to content

Object storage upload fails to Google Cloud Storage with Bucket Policy Only enabled

Summary

Uploads of artifacts (and all other types as well) fail to a Google Cloud Storage with the beta feature of Bucket Policy Only enabled.

Traditionally the Cloud IAM and the Storage ACL work together, any can grant access. This make it possible to define an ACL at upload time that allows public access to the uploaded object. No object in GitLab storages should ever be public and should only be reachable after proper authorization via a service account. The mentioned feature disables any ACL use and therefor stops this loophole of the IAM system from working.

Steps to reproduce

  • Enable the "Bucket Policy Only" setting on a storage used to upload artifacts.
  • Upload artifacts from a build.

What is the current bug behavior?

Artifact uploads fail:

WARNING: Uploading artifacts to coordinator... failed  id=115 responseStatus=500 Internal Server Error status=500 Internal Server Error token=zHEADzfx

The log shows that the upload stuff tries to set an ACL, which is forbidden in this mode:

Google::Apis::ClientError (invalid: Cannot use ACL API to set object policy when object policies are disabled.):

What is the expected correct behavior?

Uploads succeed by not setting an unneeded ACL.

Relevant logs and/or screenshots

(Paste any relevant logs - please use code blocks (```) to format console output, logs, and code as it's very hard to read otherwise.)

Output of checks

(If you are reporting a bug on GitLab.com, write: This bug happens on GitLab.com)

Results of GitLab environment info

Expand for output related to GitLab environment info 
System information
System:         Debian 9.8
Current User:   git
Using RVM:      no
Ruby Version:   2.5.5p157
Gem Version:    2.5.2.1
Bundler Version:1.17.3
Rake Version:   12.3.2
Redis Version:  3.2.6
Git Version:    2.20.1
Sidekiq Version:5.2.5
Go Version:     go1.11.5 linux/amd64

GitLab information
Version:        11.9.0-rc8
Revision:       118c883e54
Directory:      /srv/salsa-test.debian.net/gitlab
DB Adapter:     postgresql
URL:            https://salsa-test.debian.net
HTTP Clone URL: https://salsa-test.debian.net/some-group/some-project.git
SSH Clone URL:  git@salsa-test.debian.net:some-group/some-project.git
Using LDAP:     no
Using Omniauth: yes
Omniauth Providers: bitbucket, github, gitlab, google_oauth2


GitLab Shell
Version:        8.7.1
Repository storage paths:
  • default: /srv/salsa-test.debian.net/repositories GitLab Shell path: /srv/salsa-test.debian.net/gitlab-shell Git: /usr/bin/git

Possible fixes

Currently carrierwave forcefully sets stored objects to public or private, which triggers the ACL setup in fog-google.