Permissions issue on `/var/log/gitlab/mattermost`
We have reports of a crashing issue with omnibus because of permissions issue on /var/log/gitlab/mattermost
directory (please see attached).
Here's a potential workaround, need help from someone who understands omnibus internals to help stop the crashing?
Designs
- Show closed items
Activity
-
Newest first Oldest first
-
Show all activity Show comments only Show history only
@it33 That's odd; the directory should have the right owner:
On a fresh install, I don't see a problem:
$ ls -al /var/log/gitlab/ total 60 drwxr-xr-x 15 git root 4096 Jul 2 09:14 . drwxrwxr-x 16 root syslog 4096 Jul 2 06:49 .. drwx------ 2 git root 4096 Jul 2 00:34 gitlab-rails drwx------ 2 git root 4096 Jul 2 00:34 gitlab-shell drwx------ 2 git root 4096 Jul 2 09:14 gitlab-workhorse drwx------ 2 root root 4096 Jul 2 09:14 logrotate drwx------ 2 git root 4096 Jun 10 22:31 mailroom drwxr-xr-x 2 mattermost root 4096 Jul 2 09:14 mattermost drwxr-x--- 2 root gitlab-www 4096 Jul 2 09:14 nginx drwx------ 2 gitlab-psql root 4096 Jul 2 09:14 postgresql drwxr-xr-x 2 root root 4096 Jul 2 09:13 reconfigure drwx------ 2 gitlab-redis root 4096 Jul 2 09:14 redis drwx------ 2 registry root 4096 May 30 17:09 registry drwx------ 2 git root 4096 Jul 2 09:14 sidekiq drwx------ 2 git root 4096 Jul 2 09:14 unicorn
On a reconfigure, I see:
Recipe: gitlab::mattermost * group[mattermost] action create - create group mattermost * user[mattermost] action create - create user mattermost * directory[/var/opt/gitlab/mattermost] action create - create new directory /var/opt/gitlab/mattermost - change owner from '' to 'mattermost' * directory[/var/log/gitlab/mattermost] action create - create new directory /var/log/gitlab/mattermost - change owner from '' to 'mattermost' * directory[/var/opt/gitlab/mattermost/data] action create - create new directory /var/opt/gitlab/mattermost/data - change owner from '' to 'mattermost'
Working on reproing the issue again, as FYI, here's a related report
Hi @stanhu we're getting reports of this happening on the Digital Ocean one-click installer: gitlab-mattermost#77 (comment 12925760)
We tested it on a new install, on a new machine and it repros, not sure what we're missing here...
Hi,
I'm having the same issue on a fresh install of gitlab-ce 8.9.5, using docker omnibus package.
/var/log/gitlab/mattermost
owner ismattermost
, but/var/log/gitlab/mattermost/mattermost.log
owner isroot
.Changing ownership of
/var/log/gitlab/mattermost/mattermost.log
solves the issuechown mattermost /var/log/gitlab/mattermost/mattermost.log
The docker logs :
gitlab_1 | 2016-07-08_08:54:45.35098 FileLogWriter("/var/log/gitlab/mattermost/mattermost.log"): open /var/log/gitlab/mattermost/mattermost.log: permission denied gitlab_1 | 2016-07-08_08:54:45.35310 panic: runtime error: invalid memory address or nil pointer dereference [recovered] gitlab_1 | 2016-07-08_08:54:45.35314 panic: interface conversion: interface is runtime.errorString, not string gitlab_1 | 2016-07-08_08:54:45.35315 [signal 0xb code=0x1 addr=0x0 pc=0x775569] gitlab_1 | 2016-07-08_08:54:45.35318 gitlab_1 | 2016-07-08_08:54:45.35318 goroutine 1 [running]: gitlab_1 | 2016-07-08_08:54:45.35319 panic(0xcd1aa0, 0xc8201500c0) gitlab_1 | 2016-07-08_08:54:45.35319 /usr/local/go/src/runtime/panic.go:481 +0x3e6 gitlab_1 | 2016-07-08_08:54:45.35322 main.doLoadConfig.func1(0xc82017dd48) gitlab_1 | 2016-07-08_08:54:45.35323 /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:73 +0x6f gitlab_1 | 2016-07-08_08:54:45.35323 panic(0xccbec0, 0xc820012070) gitlab_1 | 2016-07-08_08:54:45.35324 /usr/local/go/src/runtime/panic.go:443 +0x4e9 gitlab_1 | 2016-07-08_08:54:45.35325 github.com/mattermost/platform/utils.configureLog(0xc8200a0c70) gitlab_1 | 2016-07-08_08:54:45.35326 /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/config.go:108 +0x539 gitlab_1 | 2016-07-08_08:54:45.35327 github.com/mattermost/platform/utils.LoadConfig(0x7ffeee862ee5, 0x26) gitlab_1 | 2016-07-08_08:54:45.35329 /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/utils/config.go:180 +0x109c gitlab_1 | 2016-07-08_08:54:45.35329 main.doLoadConfig(0x7ffeee862ee5, 0x26, 0x0, 0x0) gitlab_1 | 2016-07-08_08:54:45.35330 /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:76 +0x6a gitlab_1 | 2016-07-08_08:54:45.35331 main.main() gitlab_1 | 2016-07-08_08:54:45.35331 /var/lib/jenkins/jobs/mattermost-platform-release/workspace/src/github.com/mattermost/platform/mattermost.go:84 +0x43
My docker-compose file :
gitlab: image: gitlab/gitlab-ce:latest restart: always hostname: 'code.acme.fr' environment: GITLAB_OMNIBUS_CONFIG: | external_url 'https://code.acme.fr' gitlab_rails['time_zone'] = 'Paris' gitlab_rails['gitlab_email'] = 'gitlab@acme.fr' gitlab_rails['gitlab_email_reply_to'] = 'noreply@acme.fr' gitlab_rails['gitlab_email_display_name'] = 'GitLab Moteur' gitlab_rails['ldap_enabled'] = true gitlab_rails['ldap_servers'] = YAML.load <<-EOS # remember to close this block with 'EOS' below main: label: 'LDAP' host: 'ldap.acme.fr' port: 389 uid: 'uid' method: 'tls' bind_dn: 'bind_dn' password: 'ldap_pass' active_directory: false allow_username_or_email_login: true base: 'base' user_filter: '' EOS gitlab_rails['smtp_enable'] = true gitlab_rails['smtp_address'] = "gateway.mail.acme.fr" gitlab_rails['smtp_port'] = 25 gitlab_rails['smtp_domain'] = "code.acme.fr" registry_external_url 'https://registry.code.acme.fr' gitlab_rails['registry_enabled'] = true gitlab_rails['registry_host'] = "registry.code.acme.fr" registry_nginx['ssl_certificate'] = "/etc/gitlab/ssl/code.acme.fr.crt" registry_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/code.acme.fr.key" nginx['redirect_http_to_https'] = true nginx['ssl_client_certificate'] = "/etc/gitlab/ssl/ca-certificates.crt" nginx['ssl_certificate'] = "/etc/gitlab/ssl/code.acme.fr.crt" nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/code.acme.fr.key" mattermost_external_url 'https://mattermost.code.acme.fr/' mattermost['enable'] = true mattermost['service_use_ssl'] = true mattermost['team_site_name'] = "GitLab Moteur" mattermost_nginx['enable'] = true mattermost_nginx['redirect_http_to_https'] = true mattermost_nginx['ssl_certificate'] = "/etc/gitlab/ssl/code.acme.fr.crt" mattermost_nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/code.acme.fr.key" ports: - '$GITLAB_IP:80:80' - '$GITLAB_IP:443:443' - '$GITLAB_IP:22:22' volumes: - './volumes/config:/etc/gitlab' - './volumes/logs:/var/log/gitlab' - './volumes/data:/var/opt/gitlab' - './volumes/storage:/mnt/storage' # lots of other volumes for mounting certificates ```
- Ian Tien mentioned in issue gitlab-mattermost#77 (closed)
mentioned in issue gitlab-mattermost#77 (closed)
We have a confirmed fix with changing permissions for logfile:
Can anyone in the GitLab community help with a merge request to help fix this?
Mattermost team doesn't have expertise on omnibus.
Maybe the directory is created earlier by the runit_service (log_directory) with default uid (root)?
- DJ Mountney mentioned in merge request !898 (closed)
mentioned in merge request !898 (closed)
- DJ Mountney Reassigned to @twk3
Reassigned to @twk3
- DJ Mountney Milestone changed to %8.10
Milestone changed to %8.10
- Contributor
Fix is incoming !898 (closed)
Version check fails on on initial install, because it was running during chef cookbook convergence. Which was before the log directory, or the config was made. Then as a result of that error, the upgrade failed error message got logged using the root user, which caused the log file to be owned by the root user.
- DJ Mountney mentioned in merge request !908 (merged)
mentioned in merge request !908 (merged)
- Marin Jankovski Status changed to closed by merge request !908 (merged)
Status changed to closed by merge request !908 (merged)