After update 10.5.2: Segmentation fault at 0x0000000000000000 when doing backups and in unicorn logs

Summary

After updating GitLab from 10.4.4 to 10.5.2 doing backups errors with segmentation faults on different steps. The problem persists after updating from 10.5.2 to 10.5.3, running gitlab-ctl reconfigure and after a reboot of the OpenVZ VPS. With every execution of sudo gitlab-rake gitlab:backup:create the segfault occurs later in the process until after the ~ 5th execution the backup is finished successfully. Besides that we sporadically encounter errors when browsing the GitLab instance (viewing commits, creating projects etc.) which results in segfaults in the unicorn log at /var/log/gitlab/unicorn/unicorn_stderr.log

Steps to reproduce

• running sudo gitlab-rake gitlab:backup:create or
• browsing the GitLab web interface (it appears randomly and never happens again after a refresh)

What is the current bug behavior?

• segfault when doing backups: https://gist.github.com/adriankirchner/ccaf3265f6086f7027b5ee9c846dc924
• segfault from the unicorn error log: https://gist.github.com/adriankirchner/8a742295e55ead12ccb90bd0f62d30e7

What is the expected correct behavior?

• finishing backup jobs
• don't see GitLab errors while browsing the GitLab instance

Relevant logs

Relevant logs

- segfault while doing backups: https://gist.github.com/adriankirchner/ccaf3265f6086f7027b5ee9c846dc924
- segfault from the unicorn error log: https://gist.github.com/adriankirchner/8a742295e55ead12ccb90bd0f62d30e7

Details of package version

Provide the package version installation details

| Status=Not/Inst/Conf-files/Unpacked/halF-conf/Half-inst/trig-aWait/Trig-pend
|/ Err?=(none)/Reinst-required (Status,Err: uppercase=bad)
||/ Name                                                                                                Version                                                Architecture                                           Description
+++-===================================================================================================-======================================================-======================================================-============================================================================================================================================================================================================
ii  gitlab-ce                                                                                           10.5.3-ce.0                                            amd64                                                  GitLab Community Edition (including NGINX, Postgres, Redis)
un  gitlab-ee                                                                                                                                                                                             (no description available)

Environment details

• Operating System: Ubuntu 16.04.4 LTS
• Installation Target, remove incorrect values:
  • Other: OpenVZ VPS (Kernel 2.6.32-042stab127.2)
• Installation Type, remove incorrect values:
  • Upgrade from version 10.4.4
• Is there any other software running on the machine: no
• Is this a single or multiple node installation? single node
• Resources
  • CPU: 8 cores Intel(R) Xeon(R) CPU E5-2690 v3 @ 2.60GHz
  • Memory total: 16gb total, 11gb available

Configuration details

Provide the relevant sections of `/etc/gitlab/gitlab.rb`

external_url 'https://gitlab.mydomain.sc'
gitlab_rails['time_zone'] = 'Europe/Brussels'
gitlab_rails['gitlab_email_from'] = 'gitlab@mydomain.sc'
gitlab_rails['artifacts_enabled'] = true
gitlab_rails['backup_path'] = "/var/opt/gitlab/backups"
gitlab_rails['backup_keep_time'] = 604800
gitlab_rails['smtp_enable'] = true
gitlab_rails['smtp_address'] = "smtp.mandrillapp.com"
gitlab_rails['smtp_port'] = 587
gitlab_rails['smtp_user_name'] = "XXX"
gitlab_rails['smtp_password'] = "XXXXXXX"
gitlab_rails['smtp_domain'] = "example.com"
gitlab_rails['smtp_authentication'] = "login"
gitlab_ci['smtp_enable'] = true
gitlab_ci['smtp_address'] = "smtp.mandrillapp.com"
gitlab_ci['smtp_port'] = 587
gitlab_ci['smtp_user_name'] = "XXX"
gitlab_ci['smtp_password'] = "XXXXXXXX"
gitlab_ci['smtp_domain'] = "example.com"
gitlab_ci['smtp_authentication'] = "login"
nginx['redirect_http_to_https'] = true
nginx['redirect_http_to_https_port'] = 80
nginx['ssl_certificate'] = "/etc/gitlab/ssl/mydomain.sc.crt"
nginx['ssl_certificate_key'] = "/etc/gitlab/ssl/mydomain.sc.key"
nginx['ssl_protocols'] = "TLSv1 TLSv1.1 TLSv1.2" # recommended by https://raymii.org/s/tutorials/Strong_SSL_Security_On_nginx.html & https://cipherli.st/
nginx['custom_gitlab_server_config'] = "location ^~/.well-known/acme-challenge/ { proxy_pass http://xx.xx.xx.xx; }"
logging['logrotate_rotate'] = 7 # keep 30 rotated logs
ci_external_url 'http://ci.mydomain.sc'