12.8.1 UBI Gitaly Fails to Start

Summary

Gitaly fails to start when 12.8 UBI-based images are deployed.

The following message is shown in the kubernetes log: level=fatal msg="load config" config_path=/etc/gitaly/config.toml error="open /tmp/gitaly-linguist-path151305793: permission denied"

The issue appears to be that permissions on /tmp are set such that temp files can cannot be created by non-root users. This is causing Gitaly to fail on startup.

Steps to reproduce

  1. Create a values.yaml that specifies tags for UBI-based containers
  2. Install using helm
  3. View Gitaly logs. Error will be present.
  4. Edit Gitaly stateful set to disable liveness / readiness probes
  5. Connect to gitaly container using kubectl exec
  6. Check permissions on the /tmp folder
[git@gitlab-gitaly-0 gitaly]$ cd /
[git@gitlab-gitaly-0 /]$ ls -l
...
drwxr-xr-x   1 root root   6 Feb 24 17:33 tmp

Current behavior

Gitaly fails to start.

Expected behavior

Gitaly starts successfully.

Versions

  • Chart: 12.8.1
  • Platform:
    • Cloud: EKS
  • Kubernetes:
    • Client: 1.17.1
    • Server: v1.14.9-eks
  • Helm:
    • Client: 3.0.3
Edited by Steve Terhar