2022-10: Error Budget Monthly Report
Budget Spend Information
As per the error budget handbook page.
Please see #12 (closed) for the previous month's spend.
Generated from data on 2022-10-02 00:00:00 UTC
Recent Changes and Work In Progress
Corrupted data that was impacting the Error Budgets has been removed (see issue for details) and the spends listed below were calculated after this data was removed.
Budget Spend by Stage Group
| Product Stage | Stage Group | Error Budget | DRI | Notes |
|---|---|---|---|---|
| analytics | product_analytics | 0.00 | Not enough traffic to determine Error Budget Spend | |
| anti-abuse | anti-abuse | 0.00 | Not enough traffic to determine Error Budget Spend | |
| fulfillment | commerce_integrations | 0.00 | @isandin | Not enough traffic to determine Error Budget Spend |
| modelops | mlops | 0.00 | Not enough traffic to determine Error Budget Spend | |
| fulfillment | billing_and_subscription_management | 63.99 | @rhardarson | Last month we assigned new endpoints to this team so this team is now investigating the newly added endpoints and narrowing down which ones may be causing issues gitlab-com/runbooks!4936 (diffs). Here's a link to our Granfana dashboard. Link to Investigation issue customers-gitlab-com#4918 (comment 1123940122). |
| govern | security_policies | 84.26 | @mparuszewski | the 7-day budget is at 100%, showing that the problem that caused this drop was properly fixed. |
| modelops | applied_ml | 99.23 | @mray2020 | We launched the open beta release of Suggested Reviewer where the customer could register but not re-register after unregistering. This created lot of sidekiq errors and we plan to have the bug fixed by Oct 28th |
| monitor | observability | 99.45 | ||
| not_owned | not_owned | 99.47 | ||
| fulfillment | purchase | 99.77 | @shreyasagarwal | This is close to our target of 99.95 so I'm not too concerned but our team is still investigating to see if there are endpoints that might be concerning. Here's a link to the Grafana dashboard. https://dashboards.gitlab.net/d/stage-groups-purchase/stage-groups-purchase-group-dashboard?orgId=1&from=now-1M%2FM&to=now-1M%2FM We have created an issue to investigate the response time. #14 (moved) |
| deploy | 5-min-app | 99.82 | ||
| data_stores | database | 99.84 | ||
| govern | threat_insights | 99.87 | @thiagocsf @nmccorrison | The 7-day budget is at 99.98%, showing a positive trend. We'll continue to proactively work on performance for our categories. In particular, we're expecting another drop due to pre-existing performance issues in the Dependency Management category from before it was transferred to Threat Insights; to be addressed in gitlab#369039 (closed) Update: 2022/10/17 7-day budget now reflects 99.97%. While still a positive improvement, we have not yet eliminated our error budget over-allocation due to persisting performance issues, but efforts to improve this continue. Update: 2022/10/24 7-day budget is now 99.91%. It looks like our primary offenders are the rails API endpoint for GET /api/:version/projects/:id/vulnerability_findings, as well as our GraphQL endpoint for the vulnerabilityResolve action. Additionally it looks like there was some kind of significant service degradation on the 19th at 3:30 UTC. |
| plan | product_planning | 99.91 | ||
| create | editor | 99.93 | ||
| plan | project_management | 99.94 | ||
| secure | dynamic_analysis | 99.94 |
| Product Stage | Stage Group | Error Budget | DRI | Notes |
|---|---|---|---|---|
| data_stores | global_search | 99.86 | Exception due for review on 2022-10-30 | |
| manage | workspace | 99.86 | Exception due for review on 2022-12-31 | |
| fulfillment | provision | 66.25 | Exception due for review on 2022-10-31 |
| Product Stage | Stage Group | Error Budget | DRI | Notes |
|---|---|---|---|---|
| manage | optimize | 99.95 | ||
| secure | composition_analysis | 99.95 | ||
| verify | pipeline_authoring | 99.95 | ||
| verify | pipeline_insights | 99.95 | ||
| create | code_review | 99.95 | ||
| analytics | product_intelligence | 99.95 | ||
| create | source_code | 99.96 | ||
| growth | acquisition | 99.96 | ||
| plan | certify | 99.96 | ||
| ecosystem | foundations | 99.97 | ||
| verify | pipeline_execution | 99.97 | ||
| release | release | 99.97 | ||
| ecosystem | integrations | 99.97 | ||
| manage | import | 99.97 | ||
| manage | authentication_and_authorization | 99.98 | ||
| package | package | 99.99 | @michelletorres | |
| configure | configure | 99.99 | ||
| data_stores | pods | 99.99 | ||
| govern | compliance | 99.99 | ||
| secure | static_analysis | 99.99 | ||
| fulfillment | utilization | 99.99 | ||
| growth | activation | 100.00 | ||
| verify | runner | 100.00 | ||
| systems | gitaly | 100.00 | ||
| monitor | respond | 100.00 | ||
| fulfillment | fulfillment_platform | 100.00 | ||
| manage | compliance | 100.00 | ||
| systems | geo | 100.00 |
Edited by Michelle Torres