Skip to content

Remove Threat Insights error budget exception

Thiago Figueiró requested to merge remove-ti-eb-exception into master

Why is this change being made?

The groupthreat insights error budget exception is due for review on 2023-05-22.

This MR removes the exception. Part of gitlab-org/gitlab#383010 (closed).

Current status

  • 99.92% (28-day). Down from from from 99.94% last week.
  • 99.96% (7-day). Up from 99.89% last week.

Previous to the past 3 weeks, the error budget was stable on green. Since then, as explained in the 2023-05 error budget and infradev report, an inherited endpoint became problematic.

Justification

All the scope identified in the request to add an exception was delivered in the last quarter:

  1. Deprecate and remove Vulnerabilities::Feedback (gitlab-org&5629 - closed)
  2. Extend the use of `security_findings` (gitlab-org&8341 - closed)

These have allowed the error budget to recover, and we can confirm that the related endpoints no longer show in the kibana error budget report.

With regards to the endpoint recently identified as problematic, work to improve its performance had already begun in Group/Sub-Group Level Dependency List - MVC (gitlab-org&8090 - closed) and Use database for project dependency list (gitlab-org&8293 - closed) but it's not excepted to finish until %16.2. Until then, our error budget is susceptible to regress to red, depending on the amount of traffic the endpoint receives.

As of last week, the two heavy users of the endpoint have reduced the amount of requests, which resulted in the budget bouncing back to 99.96% in the past week.

As a backup plan, we are monitoring the budget weekly and, if necessary, will prioritize work to improve performance on the endpoint. We want to avoid doing this now because it will be throw-away work once the two epics mentioned above are delivered.

Author Checklist

  • Provided a concise title for this Merge Request (MR)
  • Added a description to this MR explaining the reasons for the proposed change, per say why, not just what
    • Copy/paste the Slack conversation to document it for later, or upload screenshots. Verify that no confidential data is added, and the content is SAFE
  • Assign reviewers for this MR to the correct Directly Responsible Individual/s (DRI)
    • If the DRI for the page/s being updated isn’t immediately clear, then assign it to one of the people listed in the Maintained by section on the page being edited
    • If your manager does not have merge rights, please ask someone to merge it AFTER it has been approved by your manager in #mr-buddies
    • The when to get approval handbook section explains the workflow in more detail
  • If the changes affect team members, or warrant an announcement in another way, please consider posting an update in #whats-happening-at-gitlab linking to this MR
    • If this is a change that directly impacts the majority of global team members, it should be a candidate for #company-fyi. Please work with internal communications and check the handbook for examples.

Edited by Phil Calder

Merge request reports