Backend: Assign custom Apdex targets for remaining PE endpoints
Context
We need to set custom Apdex targets for Rails controller and REST API endpoints as part of the initiative to improve ~SaaS reliability: gitlab-com/gl-infra/scalability#1451 (closed). In #344667 (closed), we set custom Apdex targets for CI jobs endpoints as a FY22Q4 OKR, and this issue covers the remaining grouppipeline execution endpoints.
The default SLI is 1 second. We need to determine where we will set a custom SLI (up to 5 seconds). Ideally, high traffic endpoints used directly by users (not as a part of a background process) should have a high priority set (< 1 second). We may need to set low traffic, slow endpoints to a low priority. Follow these steps to find the right priority for each endpoint: https://docs.gitlab.com/ee/development/application_slis/rails_request_apdex.html#how-to-adjust-the-urgency
Technical details
Instructions on setting custom Apdex targets: https://docs.gitlab.com/ee/development/application_slis/rails_request_apdex.html#how-to-adjust-the-urgency
Steps
-
1. Locate all grouppipeline execution REST API and controller endpoints - For now, using the Elastic dashboard is the easiest way to get to the list of endpoints. As written in the documentation. So from the Stage Group Dashboard, in the error budget attribution row, follow the "Puma Apdex" link to see a list of endpoints and the duration targets they meet or exceed.
-
2. Select a target duration for each endpoint by consulting the duration table and following the steps for determining a custom duration -
3. Where the duration we want differs from the default (1 second), add a custom target duration to the endpoint. Involve a Scalability team member in the review
Investigation table
Description | Link | Milestone | Timeframe |
---|---|---|---|
Assign custom Apdex targets for CI jobs endpoints | #344667 (closed) | %14.9 | FY22Q4 |
Set low urgency as Apdex targets for remaining PE endpoints | #360273 (closed) | %15.0 | FY23Q2 |
Remove ignored_components from PE error budget | #348552 (closed) | %15.0 | FY23Q2 |
Introduce Keyset pagination for GET /api/:version/projects/:id/jobs API endpoint |
#362172 (closed) | %15.5 | FY23Q3 |
Backend: Improve performance of GraphqlController#execute | #361377 | TBD | FY23Q3 |
Backend: Improve performance of PATCH /api/:version/jobs/:id/trace | #353802 (closed) | TBD | FY23Q3 |
Assign custom Apdex targets for remaining PE endpoints |
|
TBD | FY23Q3 |