gitlab-kas timeout during Restart Postgres on secondary Geo site
- GET version: 3.2.0
- Cloud Provider: GCP
- Environment configuration: 1k secondary Geo site
Problem
TASK [gitlab_geo : Secondary Database - Restart Postgres] **********************
fatal: [us-tiny-gotgeo-xyz-gitlab-rails-1]: FAILED! => changed=true
cmd:
- gitlab-ctl
- restart
delta: '0:00:38.041661'
end: '2024-03-01 00:07:27.968208'
msg: non-zero return code
rc: 1
start: '2024-03-01 00:06:49.926547'
stderr: ''
stderr_lines: <omitted>
stdout: |-
ok: run: geo-logcursor: (pid 66513) 1s
ok: run: geo-postgresql: (pid 66515) 0s
ok: run: gitaly: (pid 66524) 1s
ok: run: gitlab-exporter: (pid 66547) 0s
timeout: down: gitlab-kas: 0s, normally up, want up
ok: run: gitlab-workhorse: (pid 66909) 0s
ok: run: logrotate: (pid 66922) 1s
ok: run: nginx: (pid 66928) 0s
ok: run: node-exporter: (pid 66950) 0s
ok: run: postgres-exporter: (pid 66957) 1s
ok: run: postgresql: (pid 66979) 0s
ok: run: puma: (pid 66988) 1s
ok: run: redis: (pid 66993) 0s
ok: run: redis-exporter: (pid 67012) 1s
ok: run: registry: (pid 67019) 0s
ok: run: sidekiq: (pid 67032) 0s
After I SSH into the VM, I see that gitlab-kas
actually started, eventually:
run: gitlab-kas: (pid 66999) 6985s; run: log: (pid 42856) 87492s
Possible solutions
The gitlab-kas
service isn't even relevant to this part of set up. Should it specify the service to restart? gitlab-ctl restart postgresql
I'm not familiar with the gitlab-kas
service. Would it be appropriate to disable by default on a 1k secondary Geo site?
Edited by Michael Kozono