Runner API times out and 500 error with runner associated with very large number of projects
Summary
Customer has a runner associated with roughly 6000 projects and is seeing time-outs and 500 error when calling api/v4/runners/<RUNNER_ID>.
Steps to reproduce
Register a runner with a larger (possibly over 6000) projects.
What is the current bug behavior?
When trying to query a runner with a large number (over 6000) of projects, the API times out or returns 500 HTTP codes.
What is the expected correct behavior?
Query successfully returns the details of the runner.
Implementation plan
-
add an optional parameter to allow not listing all projects when getting a single runner
-
add endpoint for getting all the projects associated with runner with Pagination
diff --git a/lib/api/ci/runners.rb b/lib/api/ci/runners.rb index 42817c782f48..ffd118b6ed2d 100644 --- a/lib/api/ci/runners.rb +++ b/lib/api/ci/runners.rb @@ -141,12 +141,19 @@ def authenticate_list_runners_jobs!(runner) end params do requires :id, type: Integer, desc: 'The ID of a runner' + # TODO: deprecated, flip to false after 1X.0(17.0 probably) release, and remove in 1(X+1).0 + optional :get_projects_list, type: Boolean, desc: "Some reasonable description...", default: true # Find a better name end get ':id' do runner = get_runner(params[:id]) authenticate_show_runner!(runner) - present runner, with: Entities::Ci::RunnerDetails, current_user: current_user + present runner, with: Entities::Ci::RunnerDetails, current_user: current_user, get_projects_list: params[:get_projects_list] + end + + # params etc... + get ':id/projects' do + # return the list of projects associated with this runner WITH PAGINATION! end desc "Update runner's details" do diff --git a/lib/api/entities/ci/runner_details.rb b/lib/api/entities/ci/runner_details.rb index 8aa134dc6695..8c36aed09b3b 100644 --- a/lib/api/entities/ci/runner_details.rb +++ b/lib/api/entities/ci/runner_details.rb @@ -13,7 +13,7 @@ class RunnerDetails < Runner expose :contacted_at # rubocop: disable CodeReuse/ActiveRecord - expose :projects, with: Entities::BasicProjectDetails do |runner, options| + expose :projects, with: Entities::BasicProjectDetails, if: -> { options[:get_projects_list] } do |runner, options| if options[:current_user].can_read_all_resources? runner.projects else
-
Create a deprecation notice to warn people about flipping the default from
true
tofalse
on the next major release, **and removing the flag in the N+1.0 major release -
Create a follow-up issue for flipping the default on the next major milestone
-
Create a follow-up issue for removing the flag from the codebase on the N+1.0 major release