Lazy Load Manual Builds to avoid Request Timeouts

This issue was originally created by the sentry bot due to a RequestTimeoutException on the pipeline controller index.

https://sentry.gitlab.net/gitlab/gitlabcom/issues/1283162/

Rack::Timeout::RequestTimeoutException: Request ran for longer than 60000ms
  active_record/connection_adapters/postgresql_adapter.rb:672:in `exec_params'
    @connection.exec_params(sql, type_casted_binds)
  active_record/connection_adapters/postgresql_adapter.rb:672:in `block (2 levels) in exec_no_cache'
    @connection.exec_params(sql, type_casted_binds)
  active_support/dependencies/interlock.rb:48:in `block in permit_concurrent_loads'
    yield
  active_support/concurrency/share_lock.rb:187:in `yield_shares'
    yield
  active_support/dependencies/interlock.rb:47:in `permit_concurrent_loads'
    @lock.yield_shares(compatible: [:load]) do
...
(246 additional frame(s) were not displayed)

Rack::Timeout::RequestTimeoutException: Request ran for longer than 60000ms

Technical Proposal

Since it was found that the pipeline_controller index was spending most of it's time rendering the BuildActionEntity. We can lazy load those entities in this issue.

We can create a new REST endpoint for manual_actions and call that endpoint when the arrow is clicked. Since only one dropdown can be opened at a time in the UI, this should significantly reduce the number of jobs that will be serialized at a time and by a single endpoint. For instance, if each pipeline has 20 manual jobs to load. This leads to 300 jobs serialized ATM. After it should only load 20 jobs at a time.

For the rest of the concerns raised in this issue's comments, we can consider them in the pipeline controller index performance spike.
proposal lazy_load_manual_actions

Sequence

All behind a feature flag

  1. Add hasManualActions to the BuildActionEntity #360847 (closed)
  2. Add an endpoint that will fetch manual jobs for a single pipeline. #360847 (closed)
  3. Implement the front end loading from the new endpoint, remove references to the manual builds from the pipeline.json call #336514 (closed)
  4. Remove loading all manual builds from the pipeline index call (ensure this can be toggled via a feature flagged so that we can view the performance of the endpoint before and after the flag is enabled) #360848 (closed)
Edited by Mark Nuzzo