Allow users to specify user and group to run commands and services as
Currently we run services and commands by only specifying the user to run them as. This is based on the expectation that the primary group of a user will be same as the username. However, there are scenarios where this is not the case and customers want to specify both user and group while running services/commands. We should
- Add option to configure group for services where we already permit configuring the username
- Modify
gitlab-*
commands to use both user and group information for setting context - Pass both user and group information to chpst commands used for starting services
[Original issue description follows]
Title: Unable to upgrade gitlab to 11.2.3
Description:
Summary
After upgrading 11.2.3, I am unable to run reconfigure.
Steps to reproduce
We do not allow gitlab to manage accounts due to a requirement to use LDAP users.
manage_accounts['enable'] = false
user['username'] = "appglh"
user['group'] = "sa-git_admins"
What is the current bug behavior?
Reconfigure fails
What is the expected correct behavior?
Reconfigure should be successful
Relevant logs
Recipe: postgresql::enable[0m
* directory[/var/opt/gitlab/postgresql] action create (up to date)
* directory[/p01/app/gitlab/postgresql/data] action create (up to date)
* directory[/var/log/gitlab/postgresql] action create (up to date)
* link[/var/opt/gitlab/postgresql/data] action create (up to date)
* file[/p01/app/gitlab/postgresql/.profile] action create (up to date)
* sysctl[kernel.shmmax] action create
* directory[create /etc/sysctl.d for kernel.shmmax] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.shmmax.conf kernel.shmmax] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-kernel.shmmax.conf] action create (up to date)
* file[delete /etc/sysctl.d/90-postgresql.conf kernel.shmmax] action delete (skipped due to only_if)
* file[delete /etc/sysctl.d/90-unicorn.conf kernel.shmmax] action delete (skipped due to only_if)
* file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf kernel.shmmax] action delete (skipped due to only_if)
* file[delete /etc/sysctl.d/90-omnibus-gitlab.conf kernel.shmmax] action delete (skipped due to only_if)
* execute[load sysctl conf kernel.shmmax] action nothing (skipped due to action :nothing)
(up to date)
* sysctl[kernel.shmall] action create
* directory[create /etc/sysctl.d for kernel.shmall] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.shmall.conf kernel.shmall] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-kernel.shmall.conf] action create (up to date)
* file[delete /etc/sysctl.d/90-postgresql.conf kernel.shmall] action delete (skipped due to only_if)
* file[delete /etc/sysctl.d/90-unicorn.conf kernel.shmall] action delete (skipped due to only_if)
* file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf kernel.shmall] action delete (skipped due to only_if)
* file[delete /etc/sysctl.d/90-omnibus-gitlab.conf kernel.shmall] action delete (skipped due to only_if)
* execute[load sysctl conf kernel.shmall] action nothing (skipped due to action :nothing)
(up to date)
* sysctl[kernel.sem] action create
* directory[create /etc/sysctl.d for kernel.sem] action create (up to date)
* file[create /opt/gitlab/embedded/etc/90-omnibus-gitlab-kernel.sem.conf kernel.sem] action create (up to date)
* link[/etc/sysctl.d/90-omnibus-gitlab-kernel.sem.conf] action create (up to date)
* file[delete /etc/sysctl.d/90-postgresql.conf kernel.sem] action delete (skipped due to only_if)
* file[delete /etc/sysctl.d/90-unicorn.conf kernel.sem] action delete (skipped due to only_if)
* file[delete /opt/gitlab/embedded/etc/90-omnibus-gitlab.conf kernel.sem] action delete (skipped due to only_if)
* file[delete /etc/sysctl.d/90-omnibus-gitlab.conf kernel.sem] action delete (skipped due to only_if)
* execute[load sysctl conf kernel.sem] action nothing (skipped due to action :nothing)
(up to date)
* execute[/opt/gitlab/embedded/bin/initdb -D /p01/app/gitlab/postgresql/data -E UTF8] action run (skipped due to not_if)
* file[/p01/app/gitlab/postgresql/data/server.crt] action create (up to date)
* file[/p01/app/gitlab/postgresql/data/server.key] action create (up to date)
* template[/p01/app/gitlab/postgresql/data/postgresql.conf] action create (up to date)
* template[/p01/app/gitlab/postgresql/data/runtime.conf] action create (up to date)
* template[/p01/app/gitlab/postgresql/data/pg_hba.conf] action create (up to date)
* template[/p01/app/gitlab/postgresql/data/pg_ident.conf] action create (up to date)
* directory[/opt/gitlab/sv/postgresql] action create (up to date)
* directory[/opt/gitlab/sv/postgresql/log] action create (up to date)
* directory[/opt/gitlab/sv/postgresql/log/main] action create (up to date)
* template[/opt/gitlab/sv/postgresql/run] action create (up to date)
* template[/opt/gitlab/sv/postgresql/log/run] action create (up to date)
* template[/var/log/gitlab/postgresql/config] action create (up to date)
* ruby_block[reload postgresql svlogd configuration] action nothing (skipped due to action :nothing)
* ruby_block[restart postgresql svlogd configuration] action nothing (skipped due to action :nothing)
* file[/opt/gitlab/sv/postgresql/down] action delete (up to date)
* directory[/opt/gitlab/sv/postgresql/control] action create (up to date)
* template[/opt/gitlab/sv/postgresql/control/t] action create (up to date)
* link[/opt/gitlab/init/postgresql] action create (up to date)
* link[/opt/gitlab/service/postgresql] action create (up to date)
* ruby_block[supervise_postgresql_sleep] action run (skipped due to not_if)
* directory[/opt/gitlab/sv/postgresql/supervise] action create (up to date)
* directory[/opt/gitlab/sv/postgresql/log/supervise] action create (up to date)
* file[/opt/gitlab/sv/postgresql/supervise/ok] action touch (skipped due to only_if)
* file[/opt/gitlab/sv/postgresql/log/supervise/ok] action touch (skipped due to only_if)
* file[/opt/gitlab/sv/postgresql/supervise/control] action touch (skipped due to only_if)
* file[/opt/gitlab/sv/postgresql/log/supervise/control] action touch (skipped due to only_if)
* service[postgresql] action nothing (skipped due to action :nothing)
Recipe: postgresql::bin[0m
* ruby_block[Link postgresql bin files to the correct version] action run (skipped due to only_if)
Recipe: postgresql::enable[0m
* template[/opt/gitlab/etc/gitlab-psql-rc] action create (up to date)
* postgresql_user[gitlab] action create
* execute[create gitlab postgresql user] action run (skipped due to not_if)
(up to date)
* execute[create gitlabhq_production database] action run (skipped due to not_if)
* postgresql_user[gitlab_replicator] action create
* execute[create gitlab_replicator postgresql user] action run (skipped due to not_if)
* execute[set options for gitlab_replicator postgresql user] action run (skipped due to not_if)
(up to date)
* postgresql_extension[pg_trgm] action enable
* postgresql_query[enable pg_trgm extension] action run (skipped due to only_if)
(up to date)
* execute[reload postgresql] action nothing (skipped due to action :nothing)
* execute[start postgresql] action nothing (skipped due to action :nothing)
Recipe: gitlab::database_migrations[0m
* bash[migrate gitlab-rails database] action run
[execute] rake aborted!
Errno::EACCES: Cannot load `Rails.application.database_configuration`:
Permission denied @ rb_sysopen - /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
Errno::EACCES: Permission denied @ rb_sysopen - /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
There was an error running gitlab-ctl reconfigure:
bash[migrate gitlab-rails database] (gitlab::database_migrations line 49) had an error: Mixlib::ShellOut::ShellCommandFailed: Expected process to exit with [0], but received '1'
---- Begin output of "bash" "/tmp/chef-script20180830-40720-qwpwac" ----
STDOUT: rake aborted!
Errno::EACCES: Cannot load `Rails.application.database_configuration`:
Permission denied @ rb_sysopen - /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Caused by:
Errno::EACCES: Permission denied @ rb_sysopen - /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
/opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
/opt/gitlab/embedded/bin/bundle:23:in `load'
/opt/gitlab/embedded/bin/bundle:23:in `<main>'
Tasks: TOP => gitlab:db:configure => environment
(See full trace by running task with --trace)
STDERR:
---- End output of "bash" "/tmp/chef-script20180830-40720-qwpwac" ----
Ran "bash" "/tmp/chef-script20180830-40720-qwpwac" returned 1
[0m
================================================================================[0m
[31mError executing action `run` on resource 'bash[migrate gitlab-rails database]'[0m
================================================================================[0m
[0m Mixlib::ShellOut::ShellCommandFailed[0m
------------------------------------[0m
Expected process to exit with [0], but received '1'
[0m ---- Begin output of "bash" "/tmp/chef-script20180830-40720-qwpwac" ----
[0m STDOUT: rake aborted!
[0m Errno::EACCES: Cannot load `Rails.application.database_configuration`:
[0m Permission denied @ rb_sysopen - /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
[0m /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
[0m /opt/gitlab/embedded/bin/bundle:23:in `load'
[0m /opt/gitlab/embedded/bin/bundle:23:in `<main>'
[0m
[0m Caused by:
[0m Errno::EACCES: Permission denied @ rb_sysopen - /opt/gitlab/embedded/service/gitlab-rails/config/database.yml
[0m /opt/gitlab/embedded/service/gitlab-rails/config/environment.rb:11:in `<top (required)>'
[0m /opt/gitlab/embedded/bin/bundle:23:in `load'
[0m /opt/gitlab/embedded/bin/bundle:23:in `<main>'
[0m Tasks: TOP => gitlab:db:configure => environment
[0m (See full trace by running task with --trace)
[0m STDERR:
[0m ---- End output of "bash" "/tmp/chef-script20180830-40720-qwpwac" ----
[0m Ran "bash" "/tmp/chef-script20180830-40720-qwpwac" returned 1[0m
[0m Resource Declaration:[0m
---------------------[0m
# In /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb
[0m
[0m 49: bash "migrate gitlab-rails database" do
[0m 50: code <<-EOH
[0m 51: set -e
[0m 52: log_file="#{node['gitlab']['gitlab-rails']['log_directory']}/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log"
[0m 53: umask 077
[0m 54: /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}
[0m 55: STATUS=${PIPESTATUS[0]}
[0m 56: echo $STATUS > #{db_migrate_status_file}
[0m 57: exit $STATUS
[0m 58: EOH
[0m 59: environment env_variables unless env_variables.empty?
[0m 60: notifies :run, "execute[clear the gitlab-rails cache]", :immediately
[0m 61: dependent_services.each do |svc|
[0m 62: notifies :restart, svc, :immediately
[0m 63: end
[0m 64: not_if "(test -f #{db_migrate_status_file}) && (cat #{db_migrate_status_file} | grep -Fx 0)"
[0m 65: only_if { node['gitlab']['gitlab-rails']['auto_migrate'] }
[0m 66: end
[0m
[0m Compiled Resource:[0m
------------------[0m
# Declared in /opt/gitlab/embedded/cookbooks/cache/cookbooks/gitlab/recipes/database_migrations.rb:49:in `from_file'
[0m
[0m bash("migrate gitlab-rails database") do
[0m action [:run]
[0m default_guard_interpreter :default
[0m command nil
[0m backup 5
[0m returns 0
[0m user nil
[0m interpreter "bash"
[0m declared_type :bash
[0m cookbook_name "gitlab"
[0m recipe_name "database_migrations"
[0m code " set -e\n log_file=\"/var/log/gitlab/gitlab-rails/gitlab-rails-db-migrate-$(date +%Y-%m-%d-%H-%M-%S).log\"\n umask 077\n /opt/gitlab/bin/gitlab-rake gitlab:db:configure 2>& 1 | tee ${log_file}\n STATUS=${PIPESTATUS[0]}\n echo $STATUS > /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-06cbee3\n exit $STATUS\n"
[0m domain nil
[0m not_if "(test -f /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-06cbee3) && (cat /var/opt/gitlab/gitlab-rails/upgrade-status/db-migrate-873248b1f0d3a7a5535771a3a1635803-06cbee3 | grep -Fx 0)"
[0m only_if { #code block }
[0m end
[0m
[0m System Info:[0m
------------[0m
chef_version=13.6.4
[0m platform=redhat
[0m platform_version=7.5
[0m ruby=ruby 2.4.4p296 (2018-03-28 revision 63013) [x86_64-linux]
[0m program_name=/opt/gitlab/embedded/bin/chef-client
[0m executable=/opt/gitlab/embedded/bin/chef-client[0m
Details of package version
Omnibus Version 11.2.3 for RHEL 7 64-bit.
Environment details
- Operating System: RHEL 7 64-bit
- Installation Target, remove incorrect values:
- VM: Digital Ocean, AWS, GCP, Azure, Other
REPLACE-WITH-DETAILS
- VM: Digital Ocean, AWS, GCP, Azure, Other
- Installation Type, remove incorrect values:
- Upgrade from version 10.8.2
- Is there any other software running on the machine: None
- Is this a single or multiple node installation? Single
Additional steps taken to try and resolve.
I have attempted to change the permissions of the directories under /var/opt/gitlab/gitlab-rails. But after the reconfigure step executes and fails, it changes back the permissions. It appears during the permission change step the group/user is updated and the directory can no longer be read.
ls -l /var/opt/gitlab/gitlab-rails
total 32
drwx------. 2 appglh root 4096 Aug 30 09:10 etc
-rw-r--r--. 1 root root 8 Aug 30 08:49 REVISION
-rw-r--r--. 1 root root 58 Aug 30 08:49 RUBY_VERSION
drwxr-x---. 2 appglh sa-git_admins 4096 Aug 26 22:28 sockets
drwx------. 2 appglh root 4096 Sep 6 2017 tmp
drwx------. 2 appglh root 4096 Aug 30 08:49 upgrade-status
-rw-r--r--. 1 root root 7 Aug 30 08:49 VERSION
drwx------. 2 appglh root 4096 Aug 15 2017 working