Backend: Convert the endpoint called by MR pipelines.json to GraphQL
Problem to Solve
Per #299166 (comment 501047213), we discovered that our existing REST endpoint is taking fairly long to load due to the large amount of data being returned.
The endpoint is the one being returned by https://gitlab.com/gitlab-org/gitlab/-/merge_requests/[MR-ID]/pipelines.json. The main problem with this one is the amount of data we’re getting for each job. Then we get that data for every job in the pipeline, and every pipeline run against that MR so it compounds pretty quickly.
Technical Approach
Refactor the backend endpoint to utilize GraphQL, that way we can leverage Apollo's caching facilities reducing the overall memory footprint. Moving it to GraphQL should allow us to only request the data we need and drastically reduce the memory consumption
Other Notes
The memory consumption is concerning, but it only happens on MRs with a few large pipelines so severity2 is more appropriate, but it is certainly close to a severity1
By adding instrumentation on the pipelines table in #330051 first, it would provide some insight into where the changes should be made here and how the performance would improve.
Technical Proposal (backend)
- To the
Types::UserInterface
addshowStatus
- To the
Types::Ci::PipelineType
addsource
,flags
,name
,manualActions
,scheduledActions
- To
Types::Ci::DetailedStatusType
addillustration
- To
Types::Ci::StageType
addtitle
path
anddropdownPath