Incident workflow: auto-declare S2 deployment blocker if CNY alerts EOC
## Summary <!-- Give context for what problem this issue is trying to prevent from happening again. Provide a brief assessment of the risk (chance and impact) of the problem that this corrective action fixes, to assist with triage and prioritization. --> Currently EOC is left with an optional ~"severity::2" ~"blocks deployments" incident when CNY alerts for an Apdex degradation. This means that risky releases could potentially reach Production if manual human action isn't taken in time. This issue proposes that, following a CNY deployment, if we observe Apdex degradation, we should also auto-declare a ~"severity::2" incident which also ~"blocks deployments". This will allow EOC and release managers the time to examine the release closely to ensure safety before unblocking the deployments. When coming up with an approach to solve this, we should also do a retrospect analysis on the past data, to ensure that we're not filing too many false positives, and that the approach is working as expected without being unnecessarily noisy to those who are OnCall. ## Related Incident(s) <!-- Note the originating incident(s) and link known related incidents/other issues. The relation will happen automatically if you are creating this issue from an incident, if this isn't done already please uncomment the following line: --> Originating issue(s): gitlab-com/gl-infra/production#16032 ## Associated Services ~"Service::Woodhouse"
issue