On-Call Handover 2022-02-08 15:00 UTC

On-Call Handover

Brought to you by the Slack slash command: /sre-oncall handover

• EOC egress: iwiedler
• EOC ingress: msmiley

📖 Summary:

What (if any) time-critical work is being handed over?

What contextual info may be useful for the next few on-call shifts?

🔴 Ongoing alerts/incidents:

• production#6213 (closed) - 2022-01-23 MonitoringServiceRuleEvaluationErrorSLOViolation

• https://gitlab.com/gitlab-com/gl-infra/production/-/issues/6159

• https://gitlab.pagerduty.com/incidents/Q35XAFHJS6S2CR - [#62398] Firing 1 - Less than 100% of sentinel processes running in the redis cluster

✅ Resolved alerts/incidents:

• production#6310 (closed) - 2022-02-08: Websocket service experiencing elevated error rates

• https://gitlab.pagerduty.com/incidents/Q2EHDHZM56YFG9 - [#62403] Incident declared: Websocket service experiencing elevated error rates

🔵 Mitigated incidents:

⚪ Unactionable alerts:

🔓 Change issues:

In Progress

Closed

• production#6287 (closed) - Upgrade redis-ratelimiting on gstg to 6.2