Batch cached attribute reads in the REST runner managers list

What does this MR do and why?

Context

REST GET /runners/:id/managers reads each listed manager's cached attributes, such as version and contacted_at, with its own Redis request, one after another. A runner with many managers makes each call spend most of its time waiting on Redis. The agreed approach is in https://gitlab.com/gitlab-org/gitlab/-/work_items/638238#note_4185424743 (part 2 of 3).

This MR is stacked on !268847 (merged), which adds the batched reader for GraphQL CiRunnerManager, and targets its branch.

This MR

  • With ci_batch_rest_runner_manager_cached_attributes on, the endpoint loads the runner's managers and reads the cached attributes of all of them through Ci::RunnerManager.preload_cached_attributes before it renders them. Every cached field the entity exposes, including status, then reads the preloaded values.
  • With the flag off, each manager is read as before.
  • The response stays the same.

On a Redis Cluster the reader still sends one GET per manager, in pipelined batches, so this cuts round trips and request time, not Redis commands. The call stays unbounded in the number of managers; part 3 adds the per-user limit.

No Changelog: the change is behind a disabled flag, and the response doesn't change.

Feature Flag: ci_batch_rest_runner_manager_cached_attributes

References

Screenshots or screen recordings

Not applicable: no UI change.

How to set up and validate locally

  1. In a Rails console, create a runner with several managers and cache their attributes:
    runner = FactoryBot.create(:ci_runner)
    managers = FactoryBot.create_list(:ci_runner_machine, 5, runner: runner, version: '17.0.0')
    managers.each { |manager| manager.cache_attributes(version: '17.1.0', contacted_at: Time.current) }
    runner.id
  2. In another terminal, watch the Redis commands: redis-cli -s $(gdk config get redis.__socket_file) monitor
  3. List the managers as an administrator, with a personal access token (read_api scope):
    curl --header "PRIVATE-TOKEN: <your_access_token>" "http://gdk.test:3000/api/v4/runners/<runner_id>/managers"
    You see one GET for each manager, and version is 17.1.0.
  4. Enable the flag in the console: Feature.enable(:ci_batch_rest_runner_manager_cached_attributes)
  5. List the managers again. You see one MGET for all managers, and the same response.
  6. Clean up: Feature.remove(:ci_batch_rest_runner_manager_cached_attributes).

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Leaminn Ma

Merge request reports

Loading
Loading