Lazy Load Manual Builds to avoid Request Timeouts
This issue was originally created by the sentry bot due to a RequestTimeoutException on the pipeline controller index.
https://sentry.gitlab.net/gitlab/gitlabcom/issues/1283162/
Rack::Timeout::RequestTimeoutException: Request ran for longer than 60000ms
active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params'
@connection.exec_params(sql, type_casted_binds)
active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache'
@connection.exec_params(sql, type_casted_binds)
active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
yield
active_support/concurrency/share_lock.rb:187:in `yield_shares'
yield
active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
@lock.yield_shares(compatible: [:load]) do
...
(246 additional frame(s) were not displayed)
Rack::Timeout::RequestTimeoutException: Request ran for longer than 60000ms
Technical Proposal
Since it was found that the pipeline_controller index was spending most of it's time rendering the BuildActionEntity
. We can lazy load those entities in this issue.
We can create a new REST endpoint for manual_actions
and call that endpoint when the arrow is clicked. Since only one dropdown can be opened at a time in the UI, this should significantly reduce the number of jobs that will be serialized at a time and by a single endpoint. For instance, if each pipeline has 20 manual jobs to load. This leads to 300 jobs serialized ATM. After it should only load 20 jobs at a time.
For the rest of the concerns raised in this issue's comments, we can consider them in the pipeline controller index performance spike.
proposal | ![]() |
Sequence
All behind a feature flag
- Add
hasManualActions
to the BuildActionEntity #360847 (closed) - Add an endpoint that will fetch manual jobs for a single pipeline. #360847 (closed)
- Implement the front end loading from the new endpoint, remove references to the manual builds from the pipeline.json call #336514 (closed)
- Remove loading all manual builds from the pipeline index call (ensure this can be toggled via a feature flagged so that we can view the performance of the endpoint before and after the flag is enabled) #360848 (closed)