Gitaly service sometime not starting with Omnibus container.

Summary

Gitaly service sometime not starting with Omnibus container. Gitlab fail with 502/503 errors (/help page works).
/opt/gitlab/bin/gitlab-ctl hup gitaly fix a issue until a next time.

Steps to reproduce

docker restart gitlab several times.
Open any project page.

What is the current bug behavior?

gitaly process is not started.

What is the expected correct behavior?

Wrapper gitaly-wrapper must start gitaly

Relevant logs and/or screenshots

Correctly start:

level=info msg="Wrapper started" wrapper=521
level=info msg="finding gitaly" pid_file=/var/opt/gitlab/gitaly/gitaly.pid wrapper=521
level=info msg="spawning a process" wrapper=521
level=info msg="monitoring gitaly" gitaly=600 wrapper=521
level=info msg="Starting Gitaly" version="Gitaly, version 1.34.1"

Next incorrectly start:

level=info msg="Wrapper started" wrapper=530
level=info msg="finding gitaly" pid_file=/var/opt/gitlab/gitaly/gitaly.pid wrapper=530
level=info msg="adopting a process" wrapper=530
level=info msg="monitoring gitaly" gitaly=600 wrapper=530

As see, "adopting a process" instead "spawning a process". Wrapper use old PID 600 of a gitaly process?

Results of GitLab environment info

Expand for output related to GitLab environment info

System information
System:
Current User:	git
Using RVM:	no
Ruby Version:	2.5.3p105
Gem Version:	2.7.6
Bundler Version:1.17.3
Rake Version:	12.3.2
Redis Version:	3.2.12
Git Version:	2.18.1
Sidekiq Version:5.2.5
Go Version:	unknown
GitLab information
Version:	11.10.4
Revision:	62c464651d2
Directory:	/opt/gitlab/embedded/service/gitlab-rails
DB Adapter:	PostgreSQL
DB Version:	9.6.11
URL:		https://gitlab-ci..org
HTTP Clone URL:	https://gitlab-ci..org/some-group/some-project.git
SSH Clone URL:	git@gitlab-ci.*****.org:some-group/some-project.git
Using LDAP:	no
Using Omniauth:	yes
Omniauth Providers:
GitLab Shell
Version:	9.0.0
Repository storage paths:

• default: /var/opt/gitlab/git-data/repositories GitLab Shell path: /opt/gitlab/embedded/service/gitlab-shell Git: /opt/gitlab/embedded/bin/git

Results of GitLab application Check

Expand for output related to GitLab check

Checking GitLab subtasks ...
Checking GitLab Shell ...
GitLab Shell: ... GitLab Shell version >= 9.0.0 ? ... OK (9.0.0)
Running /opt/gitlab/embedded/service/gitlab-shell/bin/check
Check GitLab API access: OK
Redis available via internal API: OK
Access to /var/opt/gitlab/.ssh/authorized_keys: OK
gitlab-shell self-check successful
Checking GitLab Shell ... Finished
Checking Gitaly ...
Gitaly: ... default ... OK
Checking Gitaly ... Finished
Checking Sidekiq ...
Sidekiq: ... Running? ... yes
Number of Sidekiq processes ... 1
Checking Sidekiq ... Finished
Checking Incoming Email ...
Incoming Email: ... Reply by email is disabled in config/gitlab.yml
Checking Incoming Email ... Finished
Checking LDAP ...
LDAP: ... LDAP is disabled in config/gitlab.yml
Checking LDAP ... Finished
Checking GitLab App ...
Git configured correctly? ... yes
Database config exists? ... yes
All migrations up? ... yes
Database contains orphaned GroupMembers? ... no
GitLab config exists? ... yes
GitLab config up to date? ... yes
Log directory writable? ... yes
Tmp directory writable? ... yes
Uploads directory exists? ... yes
Uploads directory has correct permissions? ... yes
Uploads directory tmp has correct permissions? ... yes
Init script exists? ... skipped (omnibus-gitlab has no init script)
Init script up-to-date? ... skipped (omnibus-gitlab has no init script)
Projects have namespace: ...
2/1 ... yes
3/2 ... yes
2/3 ... yes
6/5 ... yes
6/6 ... yes
5/7 ... yes
6/8 ... yes
2/9 ... yes
6/11 ... yes
18/14 ... yes
18/15 ... yes
18/18 ... yes
5/19 ... yes
5/20 ... yes
5/21 ... yes
5/22 ... yes
22/23 ... yes
22/24 ... yes
20/27 ... yes
5/28 ... yes
19/29 ... yes
17/30 ... yes
18/31 ... yes
17/32 ... yes
25/34 ... yes
5/35 ... yes
19/36 ... yes
23/37 ... yes
23/38 ... yes
19/39 ... yes
5/40 ... yes
31/41 ... yes
31/42 ... yes
31/43 ... yes
3/46 ... yes
24/47 ... yes
21/48 ... yes
20/49 ... yes
32/50 ... yes
12/51 ... yes
21/52 ... yes
23/54 ... yes
37/55 ... yes
33/56 ... yes
34/57 ... yes
35/61 ... yes
35/62 ... yes
39/63 ... yes
39/64 ... yes
42/67 ... yes
Redis version >= 2.8.0? ... yes
Ruby version >= 2.5.3 ? ... yes (2.5.3)
Git version >= 2.18.0 ? ... yes (2.18.1)
Git user has default SSH configuration? ... yes
Active users: ... 17
Checking GitLab App ... Finished
Checking GitLab subtasks ... Finished

Possible fixes

Unfortunately, I didn’t find the gitaly-wrapper source code to understand the reason.
Probably, at launch time there is some other process with the PID of the previous gitaly (600 in this example). And a wrapper considers it already running.