Commit 5d615813 authored by Florbela Viegas's avatar Florbela Viegas 😀 Committed by Steve Abrams
Browse files

Align severities of incidents and infradev to business impact for escalated...

Align severities of incidents and infradev to business impact for escalated customers and Dedicated fleet operability
parent b4008b4f
Loading
Loading
Loading
Loading
+2 −2
Changes for content/handbook/engineering/infrastructure-platforms/incident-management/_index.md: 2 added lines, 2 removed lines.
Original line number Diff line number Diff line
@@ -564,8 +564,8 @@ Incident Managers and Engineers On-Call can use the following table as a guide f

| Severity | Impact | GitLab Response | Examples |
|--------|-------------|-------------|---------------------|
| Severity:1 **Critical** | **Customer Impact:** <br> Very high impact on users: their customers or business outputs will be impacted <br><br> **OR** <br><br> **GitLab Impact:** <br> Probable or severe damage to the business |Immediate all-hands response | - Customer-facing service is down <br> - Confirmed data breach or exposure of red/orange data <br> - Customer data loss <br> - Low-complexity, validated exploit scenario to GitLab’s platform or supply chain. <br> - Critical RCE that is actively exploited or that is unpatched, reachable, and has no exploitability telemetry <br> - Critical vulnerability that has public exposure (press, customers, 0-day by researcher) <br> - External actor controls a highly privileged GitLab service account|
| Severity:2 **High** | **Customer Impact:** <br> Significant impact on users: their internal operations will be disrupted <br><br> **OR** <br><br>**GitLab Impact:** <br> Possible or elevated damage to the business | Assigned resources, cross-team coordination, and regular stakeholder updates | - Customer-facing service is unavailable for some customers<br> - Core functionality is significantly impacted<br> - Privilege escalation scenarios requiring account compromise or insider-threat motive and knowledge<br>- High severity vulnerability with evidence of exploitation OR high press attention<br>- Suspected unauthorized access into sensitive GitLab systems<br>- Malware detection in GitLab's cloud infrastructure |
| Severity:1 **Critical** | **Customer Impact:** <br> Very high impact on users: their customers or business outputs will be impacted <br><br> **OR** <br><br> **GitLab Impact:** <br> Probable or severe damage to the business: <br> Dedicated: <br>    - Severe disruption to scale and operability in Dedicated fleet <br>      OR <br>      - High attrition risk of escalated customers in Dedicated with visible impact (small or large) . |Immediate all-hands response | - Customer-facing service is down <br> - Confirmed data breach or exposure of red/orange data <br> - Customer data loss <br> - Low-complexity, validated exploit scenario to GitLab’s platform or supply chain. <br> - Critical RCE that is actively exploited or that is unpatched, reachable, and has no exploitability telemetry <br> - Critical vulnerability that has public exposure (press, customers, 0-day by researcher) <br> - External actor controls a highly privileged GitLab service account <br> - Dedicated business impact: Blocked ability to change or scale fleet in an automated way, high accumulation of version drift and manual operations <br> - Dedicated business impact: Saturation of response capacity, degraded response quality to critical customers, especially high risk escalated customers <br> - Risk of missing the monthly self-managed minor or major release |
| Severity:2 **High** | **Customer Impact:** <br> Significant impact on users: their internal operations will be disrupted <br><br> **OR** <br><br>**GitLab Impact:** <br> Possible or elevated damage to the business: <br> Dedicated : <br> - Fleet rollout blocked by fixes for more than a week <br> OR <br>  - High attrition risk of escalated customers in Dedicated without visible product impact | Assigned resources, cross-team coordination, and regular stakeholder updates | - Customer-facing service is unavailable for some customers<br> - Core functionality is significantly impacted<br> - Privilege escalation scenarios requiring account compromise or insider-threat motive and knowledge<br>- High severity vulnerability with evidence of exploitation OR high press attention<br>- Suspected unauthorized access into sensitive GitLab systems<br>- Malware detection in GitLab's cloud infrastructure |
| Severity:3 **Medium** | **Customer Impact:** <br> Moderate impact on users: their internal operations may be hampered <br><br> **OR** <br><br> **GitLab Impact:** <br> Unlikely or mild damage to the business | Resources are diverted to address beyond normal operating procedures | - Slight performance degradation<br>- Non-critical features not performing optimally<br>- Commodity malware detection in non-critical systems  |
| Severity:4 **Low** | **Customer Impact:**: <br> Low impact on users: their internal operations may be altered <br><br> **OR** <br><br> **GitLab Impact:** Minimal damage to the business | Issue is resolved following standard procedures | - An inconvenience to customers, workaround available<br>- Usable performance degradation<br>- GitLab security policy violations that do not impact red/orange data  |

+2 −2
Changes for content/handbook/product-development/how-we-work/issue-triage.md: 2 added lines, 2 removed lines.
Original line number Diff line number Diff line
@@ -151,8 +151,8 @@ The presence of these severity labels modifies the standard severity labels(`~"s

| Severity | Availability impact | Time to mitigate (TTM)(1) | Time to resolve (TTR)(2) | Minimum priority |
|-|-|-|-|-|
| `~"severity::1"` | Problem on GitLab SaaS blocking the typical user's workflow<br/><br/>Impacts 20% or more of users in GitLab.com or a tenant on Dedicated, without an available workaround<br/><br/>**AND/OR**<br/><br/>Any roadblock that puts the [guaranteed self-managed release date](/handbook/engineering/releases/monthly-releases/#timelines) at risk (use ~backstage label)<br /><br/>**AND/OR**<br/><br/>Any data loss directly impacting customers <br/>**AND/OR**<br/> Causes repeated incidents on SaaS hindering our ability to onboard new customers | Within 8 hrs | Within 48 hrs | `~"priority::1"` |
| `~"severity::2"` | Problem on GitLab SaaS blocking the typical user's workflow<br/><br/>Impacts 20% or more of users on GitLab.com or a tenant on Dedicated, but a reasonable workaround is available.<br/><br/>Impacts between 5%-20% of users on GitLab.com or a tenant on Dedicated without an available workaround | Within 24 hrs | Within 7 days |  `~"priority::1"` |
| `~"severity::1"` | Problem on GitLab SaaS blocking the typical user's workflow<br/><br/>Impacts 20% or more of users in GitLab.com or a tenant on Dedicated, without an available workaround<br/><br/>**AND/OR**<br/><br/>Any roadblock that puts the [guaranteed self-managed release date](/handbook/engineering/releases/monthly-releases/#timelines) at risk (use ~backstage label)<br/><br/>**AND/OR**<br/><br/>Any data loss directly impacting customers<br/><br/>**AND/OR**<br/><br/>Causes repeated incidents on SaaS, putting Dedicated fleet operations at risk and hindering scale through excess manual interventions<br/><br/>**AND/OR**<br/><br/>Issue is blocking automated fleet rollouts in Dedicated, delaying scheduled releases for more than one week | Within 8 hrs | Within 48 hrs | `~"priority::1"` |
| `~"severity::2"` | Problem on GitLab SaaS blocking the typical user's workflow<br/><br/>Impacts 20% or more of users on GitLab.com or a tenant on Dedicated, but a reasonable workaround is available.<br/>**AND/OR**<br/>Impacts between 5%-20% of users on GitLab.com or a tenant on Dedicated without an available workaround | Within 24 hrs | Within 7 days |  `~"priority::1"` |
| `~"severity::3"` | Broad impact on GitLab SaaS and minor inconvenience to typical user's workflow. No workaround needed.<br/><br/>Impacts up to 5% of users on GitLab.com or a tenant on Dedicated| Within 72 hrs | Within 30 days | `~"priority::2"` |
| `~"severity::4"` | Minimal impact on GitLab SaaS typical user's workflow to less than 5% of users on GitLab.com or a tenant on Dedicated <br/><br/>May also include incidents with no impact, but with importance to resolve to prevent future risk| Within 7 days | Within 60 days | `~"priority::3"` |