Batch cached attribute reads for CiRunnerManager
What does this MR do and why?
Fields of GraphQL CiRunnerManager that read cached attributes (version, status, contactedAt, upgradeStatus and others) made one Redis GET for each manager on the page. A request that reaches 5,200 managers made 5,200 GETs, one after another, and took 4 to 10 seconds in Redis calls.
This MR reads the cached attributes of all managers on the page together:
RedisCacheable::CachedAttributesReaderholds the cached attributes of one record and reads them from Redis at most once.CachedAttributesReader.preloadfills many readers with one batched read. On a Redis Cluster it usesGitlab::Redis::ClusterUtil.batch_get. On other Redis it usesMGET.RedisCacheable.preload_cached_attributespreloads the readers of many records.Types::Ci::RunnerManagerTypeand the EEupgradeStatusread cached attributes throughwith_cached_attributes. It uses aBatchLoaderthat preloads the managers that GraphQL returns. The batch covers only the managers on the page, not every manager that the resolver loaded.- The change is behind the
gitlab_com_deriskfeature flagci_batch_runner_manager_cached_attributes, which is disabled by default. With the flag off, each manager is read as before. with_cached_attributesyields the preloaded manager. A query can load the same manager twice, so that manager can be another instance of the same record.
On GitLab.com the keys are in different hash slots, so MGET is not possible. Redis still runs one GET for each manager, but the app sends them in a few pipelined batches instead of waiting for each reply. This lowers round trips and request time. It does not lower the number of Redis commands. The effect on Redis CPU is measured during rollout.
Not in this MR: RunnerManagersResolver still loads every manager of the requested runners from the database and slices the page in Ruby. The assessment tracks this separately.
Feature flag
- Flag:
ci_batch_runner_manager_cached_attributes(gitlab_com_derisk, disabled by default) - Rollout issue: #639071
- Removal:
config/feature_flags/gitlab_com_derisk/ci_batch_runner_manager_cached_attributes.patchremoves the flag check and the specs for the disabled flag. TheDeleteOldFeatureFlagskeep applies it when it removes the flag.
References
- Work item: https://gitlab.com/gitlab-org/gitlab/-/work_items/638242
- Parent: https://gitlab.com/gitlab-org/gitlab/-/work_items/638233
- Rollout issue: #639071
- Assessment: https://gitlab.com/gitlab-org/gitlab/-/work_items/636925#note_4126705820
Screenshots or screen recordings
Not applicable. There is no UI change.
How to set up and validate locally
- In a Rails console, create a runner with several managers and cache their attributes:
runner = FactoryBot.create(:ci_runner) managers = FactoryBot.create_list(:ci_runner_machine, 5, runner: runner, version: '17.0.0') managers.each { |manager| manager.cache_attributes(version: '17.1.0') } - In another terminal, watch the Redis commands:
redis-cli -s $(gdk config get redis.__socket_file) monitor - Run this query in GraphiQL at
http://127.0.0.1:3000/-/graphql-explorer:You see one{ runners(first: 100) { nodes { managers(first: 100) { nodes { version status } } } } }GETfor each manager, andversionis17.1.0. - Enable the flag in the console:
Feature.enable(:ci_batch_runner_manager_cached_attributes) - Run the query again. You see one
MGETfor all managers, and the same result.
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.