Support consolidated object store configuration in Charts
Summary
Add support of the consolidated object storage within charts via global.appConfig.object_store
. Have workhorse consume this to configure it's own object_storage(.s3)
settings as well.
Implementation
Adds global.appConfig.object_store
configuration to the charts. This allows the the configuration of a single connection
property, which will be consumed by Rails (Sidekiq, Webservice, Task-runner) to configure lfs, uploads, artifacts, ( ... ). This does not include backups, as consumed by task-runner. That is not integrated with the Rails/Rake codebase, and as such does not currently have the capability.
This is implemented in such a fashion that the included MinIO (when used) will be propagated in this way. When not using an external object storage provider, this will result in a single /etc/gitlab/objectstorage/object_store
connection file, which Rails itself will consume & populate into all the individual configurations.
When using external storage, such as GCS, Amazon S3, or Azure blob storage (coming soon) this will also be the case.
Noticable difference
Configuring object storage can now be much simpler than as previously, where the settings need to be used with YAML anchors or copy-pasted when the connection configuration / secrets where identical
object_store:
enabled: true
proxy_download: true # default
connection:
secret: my-aws-secret
key: connection
artifacts:
enabled: true # default, on/off of feature
bucket: gitlab-artifacts # default
uploads:
enabled: true # default, on/off of feature
bucket: gitlab-uploads # default
lfs:
enabled: true # default, on/off of feature
bucket: gitlab-lfs # default
provider: AWS
aws_access_key_id: AWS_ACCESS_KEY_ID
aws_secret_access_key: AWS_SECRET_ACCESS_KEY
region: us-east-1
When using the provider: AWS
, upload handling can be handled entirely by gitlab-workhorse
, without reliance on pre-signed URLs from gitlab-webservice
. This brings direct support for Content-MD5
header, which should address problems with ETag mismatch and bring greater support for AWS S3 bucket encryption.
Note: Only S3 compatible object storage providers are supported in combination with workhorse offloading currently.
Closes #2158 (closed)