Allow for a configurable log group for log dirs and runit-managed logs
What does this MR do?
This implements functionality to allow for a global (using logging['log_group']
) or per-service (e.g. gitaly['log_group']
) configuration setting that will be set as the group on Omnibus-managed log directories; will be used as the execution group for launching svlogd with chpst for the runit-managed services in Omnibus; and will be the group attached to the rotated log files for logrotate (making use of the su
directive)
While this MR does implement code for passing in a log_user
as well, that configuration is not exposed, and the user/owner is left hardcoded as root
for runit/svlogd-managed logs, and the current user/owner that's already set for the log directories in order to prevent any unexpected behavior by changing the log directory owner (e.g. svlogd fails when an existing root-owned lock
file is present, and is switched using chpst
to another user, without changing the permissions on that lock
file, as well as other process tracking files)
By default, the existing behavior is preserved - Omnibus recipes continue to set the log directories with the harcoded and/or configurable usernames (root
, git
, gitlab-psql
, gitlab-redis
, etc.) - and 0700
permissions.
When a log_group
is specified globally and/or per-service:
- The associated
log_directory
with the service(s) have their group set tolog_group
and mode0750
- For runit-managed services, the
log/run
template used by runit to start svlogd execschpst
with-U/-u root:configured_log_group
- For logrotate - the configured log_group will be used as the group in the
create
statement for rotated logs
This should allow members of the log_group
to walk into the log directories - and will allow that group & members to read the svlogd written log.
This MR only handles svlogd-managed logs directly - it makes no current attempt to change the permissions on other logs, and does not change the permissions on any existing rotated logs.
Related issues
Closes Feature Proposal: Omnibus Configuration to Adju... (#6731 - closed)
Closes Make log directory creation by service recipes ... (#4606 - closed)
Also provides a partial solution for gitlab#1527
Checklist
See Definition of done.
For anything in this list which will not be completed, please provide a reason in the MR discussion
Required
-
Merge Request Title, and Description are up to date, accurate, and descriptive -
MR targeting the appropriate branch -
MR has a green pipeline on GitLab.com -
Pipeline is green on dev.gitlab.org if the change is touching anything besides documentation or internal cookbooks -
trigger-package
has a green pipeline running against latest commit
Expected (please provide an explanation if not completing)
-
Test plan indicating conditions for success has been posted and passes -
Documentation created/updated -
Tests added -
Integration tests added to GitLab QA -
Equivalent MR/issue for the GitLab Chart opened
Test Plan
Stand up an Omnibus install with this branch ( Packages are available in the latest pipeline: gitlab-ee | gitlab-ce ) and:
- Create or identify an existing group(s) on the host to use as the
log_group
- Set
logging['log_group']
to one of those identified/created groups - Run
sudo gitlab-ctl reconfigure
- Observe that the permissions on
/var/log/gitlab/[service]
are now configured with the specifiedlog_group
and mode0750
- Observe that
svlogd
processes are started withroot:log_group
- and the group assigned to/var/log/gitlab/[runit-managed-service]/current
islog_group
Comment out the logging['log_group']
setting and observe that no groups are modified for the /var/log/gitlab/[service]
log_directories, but that the mode is reset to the default 0700
setting. All svlogd
-managed current
logs will also revert to root:root
- and root:root
will be used for the chpst
execution of svlogd
Additional Notes
This MR contains a huge number of changes due to the number of services involved, but the vast majority of it is repetition for log directory creation - as well as per-service tests for the log_directory and chpst settings (this MR corrects a deficiency that not all services were testing for the permissions/mode on the log_directory)
As a result of this comment - I've gone back and made use of the helper in the logrotate
recipe - renaming it to LogfilesHelper
to expand upon its use for permissions on log directories, logrotate, and for pulling out a runit owner/group for use with chpst to start svlogd - this allowed for addressing #4606 (closed) as well.
However, the foundational goal of this MR is to very intentionally preserve existing behavior and configuration unless a global or per-service log_group
is supplied - which leads to additional complexity in order to distinguish between the group used for the log directory, the group used for logrotate, and the group used for runit/svlogd.
The current behavior is:
- For everything except
nginx
- Omnibus does not change the group attached to a log_directory - the group assigned to a log directory isnil
. The nginx service does usegitlab-www
(note: - the docker container has an update-permissions script that will assign groups to log_directories) - For everything except
consul
,mattermost
, andnginx
- the permissions on the log_directory directories are reset to '0700'. Thenginx
service will use0750
-consul
andmattermost
use0755
- For logrotate, Omnibus will use a default/hardcoded group for the
su user group
directive in thelogrotate.d
configuration for the service - unless theservice['group']
configuration option is passed in. - All runit-managed services exec
svlogd
asroot:root
- androot:root
ends up being the owner:group attached to both thecurrent
log, as well as any svlogd-rotated logs
This MR preserves all the current behaviors unless a global or per-service log_group
is explicitly set - that log_group
will be used as the log_directory group, the logrotate
group, and the execution group for svlogd
.