2018-07-16: Intermittent ssl errors on pages when enabling the healthcheck
Summary
On July 16th there were intermittent pages errors for approximately 5 minutes.
The root cause was the result of a configuration change that added the readiness health check. gitlab-com/migration#647 (closed) Although all backends were reporting up we were seeing ssl errors on pages after applying the following change:
backend pages_https
mode tcp
option splice-auto
balance source
hash-type consistent
- option httpchk GET /-/readiness HTTP/1.1\r\nHost:\ gitlab.com
- server web01.fe.gitlab.com 10.69.8.101:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web02.fe.gitlab.com 10.69.8.102:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web03.fe.gitlab.com 10.69.8.103:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web04.fe.gitlab.com 10.69.8.104:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web05.fe.gitlab.com 10.69.8.105:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web06.fe.gitlab.com 10.69.8.106:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web07.fe.gitlab.com 10.69.8.107:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web08.fe.gitlab.com 10.69.8.108:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web09.fe.gitlab.com 10.69.8.109:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web10.fe.gitlab.com 10.69.8.110:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web11.fe.gitlab.com 10.69.8.111:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web12.fe.gitlab.com 10.69.8.112:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web13.fe.gitlab.com 10.69.8.113:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
- server web14.fe.gitlab.com 10.69.8.114:1443 check check-ssl inter 2s fastinter 1s downinter 5s fall 3 ssl verify none
+ server web01.fe.gitlab.com 10.69.8.101:1443 check inter 5000 fall 3
+ server web02.fe.gitlab.com 10.69.8.102:1443 check inter 5000 fall 3
+ server web03.fe.gitlab.com 10.69.8.103:1443 check inter 5000 fall 3
+ server web04.fe.gitlab.com 10.69.8.104:1443 check inter 5000 fall 3
+ server web05.fe.gitlab.com 10.69.8.105:1443 check inter 5000 fall 3
+ server web06.fe.gitlab.com 10.69.8.106:1443 check inter 5000 fall 3
+ server web07.fe.gitlab.com 10.69.8.107:1443 check inter 5000 fall 3
+ server web08.fe.gitlab.com 10.69.8.108:1443 check inter 5000 fall 3
+ server web09.fe.gitlab.com 10.69.8.109:1443 check inter 5000 fall 3
+ server web10.fe.gitlab.com 10.69.8.110:1443 check inter 5000 fall 3
+ server web11.fe.gitlab.com 10.69.8.111:1443 check inter 5000 fall 3
+ server web12.fe.gitlab.com 10.69.8.112:1443 check inter 5000 fall 3
+ server web13.fe.gitlab.com 10.69.8.113:1443 check inter 5000 fall 3
+ server web14.fe.gitlab.com 10.69.8.114:1443 check inter 5000 fall 3
Edited by John Jarvis