Gitlab-in-docker requires different/unclear permissions for different credentials/config files
Summary
I run an instance of the gitlab-ee
docker image. I have set it up to use Google Cloud Storage for object storage, using the recommended consolidated form.
As a part of the normal setup, you specify a path to a json that contains GCS credentials for accessing the buckets:
gitlab_rails['object_store']['connection'] = {
'provider' => 'Google',
'google_project' => 'redacted',
'google_client_email' => 'redacted@redacted',
'google_json_key_location' => '/etc/gitlab/gitlab-gcs-credentials.json'
}
My problem is that it is not clear what permissions I should be setting for this json file google_json_key_location
.
If I use the following permissions for this json, all cloud storage operations fail (notably, these permissions are the same as gitlab.rb
and gitlab-secrets
):
$ docker exec gitlab ls -lh /etc/gitlab/
-rw------- 1 root root 2.3K Oct 24 00:09 gitlab-gcs-credentials.json
-rw------- 1 root root 19K Oct 25 00:11 gitlab-secrets.json
-rw------- 1 root root 81K Oct 25 00:11 gitlab.rb
(some entries removed)
I see the following error appear in the log when attempting to do anything that interacts with the buckets (e.g. uploading an image attachment to a comment on an issue) -
==> /var/log/gitlab/gitlab-rails/production.log <==
Errno::EACCES (Permission denied @ rb_sysopen - /etc/gitlab/gitlab-gcs-credentials.json):
lib/object_storage/direct_upload.rb:231:in `connection'
lib/object_storage/direct_upload.rb:129:in `get_url'
lib/object_storage/direct_upload.rb:44:in `to_hash'
app/uploaders/object_storage.rb:211:in `workhorse_remote_upload_options'
app/uploaders/object_storage.rb:185:in `block in workhorse_authorize'
app/uploaders/object_storage.rb:183:in `tap'
app/uploaders/object_storage.rb:183:in `workhorse_authorize'
app/controllers/concerns/uploads_actions.rb:58:in `authorize'
ee/lib/gitlab/ip_address_state.rb:10:in `with'
ee/app/controllers/ee/application_controller.rb:44:in `set_current_ip_address'
However, if I change the permissions to 644
(global read):
$ docker exec gitlab ls -lh /etc/gitlab/
-rw-r--r-- 1 root root 2.3K Oct 24 00:09 gitlab-gcs-credentials.json
-rw------- 1 root root 19K Oct 25 00:11 gitlab-secrets.json
-rw------- 1 root root 81K Oct 25 00:11 gitlab.rb
... the problem no longer occurs. I can go into the buckets in the GCS interface and see files appearing.
I would much rather not have to use 644
permissions for this secrets file, as it is an obvious security risk. However, permissions 640
and 600
both do not work.
I would assume that I need to change the owner of the file to one of the gitlab-XXX
users that are present on the system, however it is not clear/not documented who should own the file. Though, given that this file contains secrets, this file would ideally be owned by root (similar to gitlab-secrets
and other sensitive instance files).
Steps to reproduce
Set up a gitlab instance with google cloud storage as an object storage backend. Set the permissions of the google_json_key_location
file to owner root:root
and access 640
or 600
. Observe that the system cannot read the json, and fails to execute any cloud storage operations.
Example Project
(this is on my own instance, I can't demonstrate this anywhere)
What is the current bug behavior?
I am required to set 644
permissions on the google_json_key_location
file, or required to set the owner of the file to some user that is not documented anywhere.
What is the expected correct behavior?
I would be able to use root:root 600
permissions on the file, same as gitlab.rb
and gitlab-secrets.json
, which I would classify as equally sensitive/critical.
Relevant logs and/or screenshots
logs are in-line with relevant sections in the summary section
Output of checks
?
Results of GitLab environment info
$ docker exec gitlab_gitlab_1 gitlab-rake gitlab:env:info
System information
System:
Proxy: no
Current User: git
Using RVM: no
Ruby Version: 2.7.4p191
Gem Version: 3.1.4
Bundler Version:2.1.4
Rake Version: 13.0.6
Redis Version: 6.0.14
Git Version: 2.33.0.
Sidekiq Version:6.2.2
Go Version: unknown
GitLab information
Version: 14.4.0-ee
Revision: 19c719febd7
Directory: /opt/gitlab/embedded/service/gitlab-rails
DB Adapter: PostgreSQL
DB Version: 12.7
URL: https://dont-want-to-disclose
HTTP Clone URL: https://dont-want-to-disclose/some-group/some-project.git
SSH Clone URL: git@dont-want-to-disclose:some-group/some-project.git
Elasticsearch: no
Geo: no
Using LDAP: no
Using Omniauth: yes
Omniauth Providers: gitlab, google_oauth2
GitLab Shell
Version: 13.21.1
Repository storage paths:
- default: /var/opt/gitlab/git-data/repositories
GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell
Git: /opt/gitlab/embedded/bin/git
Results of GitLab application Check
$ docker exec gitlab_gitlab_1 gitlab-rake gitlab:check SANITIZE=true
Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 13.21.1 ? ... OK (13.21.1)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Internal API available: OK
Redis available via internal API: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes (cluster/worker) ... 1/1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
42/2 ... yes
34/3 ... yes
94/6 ... yes
<... snip, it's all yes ...>
238/608 ... yes
235/609 ... yes
224/612 ... yes
Redis version >= 5.0.0? ... yes
Ruby version >= 2.7.2 ? ... yes (2.7.4)
Git version >= 2.33.0 ? ... yes (2.33.0)
Git user has default SSH configuration? ... yes
Active users: ... 28
Is authorized keys file accessible? ... yes
GitLab configured to store new projects in hashed storage? ... yes
All projects are in hashed storage? ... yes
Elasticsearch version 7.x (6.4 - 6.x deprecated to be removed in 13.8)? ... skipped (elasticsearch is disabled)
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished