Draft: [skip ci] Add the ciStageJobsUpdated subscription
What does this MR do and why?
ciStageUpdated sends the job that changed, one update for each job transition. Its only consumer is the stage dropdown of the pipeline mini graph. A stage with hundreds of jobs then pushes hundreds of updates to an open dropdown, and the coalescer from !266624 (merged) cannot merge them, because each update is about a different job. In one MR pipeline, one stage got 668 triggers in a minute.
This MR adds ciStageJobsUpdated, which sends the stage instead:
- Each update is the current state of all jobs in the stage. The client selects the jobs it needs, as in
getPipelineStageJobs. - The subscription declares
coalesce, with the default window of 500 milliseconds. A burst of job transitions in one stage becomes at most one update for each window, and the last one always arrives. - It reuses
Subscriptions::Ci::Stages::StatusUpdatedfor the arguments, authorization andupdate, and returnsCiStage, likeciStageStatusUpdated.ciStageStatusUpdateditself is not coalesced.
The trigger
- On every job type. The trigger is on
CommitStatus, notCi::Build. Bridges and generic commit statuses now update their stage too.ciStageUpdatednever covered them, so a trigger job in an open dropdown did not update. - No query. The trigger runs on every job transition, so it must not load the stage. It publishes the stage by its global ID through
Gitlab::Graphql::Subscriptions::RecordReference, and the websocket nodes load the stage when they run the subscription. The topic argument has to be aGlobalID, not theURI::GIDthatGitlab::GlobalId.buildreturns, because subscribers' topics hold its base64 form. The trigger spec checks both.
Rollout
The field is an experiment. The stack around it:
- This MR: the new field and trigger. Until the removal MR, both triggers publish, which doubles the Redis messages for stage updates.
- !266647 moves the mini graph to the new field.
- !266648 removes
ciStageUpdatedand its trigger, about 750 Redis messages each second on GitLab.com.
Merge the three close together, so the double publishing lasts as short as possible.
How to set up and validate locally
bin/rspec spec/graphql/subscriptions/ci/stages spec/graphql/graphql_triggers_spec.rb \
spec/graphql/types/subscription_type_spec.rb spec/models/commit_status_spec.rbVerified in GDK
With !266647 on top, the stage dropdown of a pipeline in gitlab-org/gitlab-test subscribed to ciStageJobsUpdated for its test stage (201 jobs). From a Rails runner, GraphqlTriggers.ci_stage_jobs_updated(job) was called 10 times within a few milliseconds:
graphql_subscription_coalescing |
Results sent to the browser |
|---|---|
| Off | 11 |
| On | 3 |
The pipeline was running, so each run also included one real job transition, outside the burst. Each result carried the stage with its jobs, and the browser console had no errors.
![]() |
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist.
References
- Part of https://gitlab.com/gitlab-org/gitlab/-/work_items/630854, step C3
- Uses the coalescer from !266624 (merged)
- Frontend switch: !266647
- Removal of
ciStageUpdated: !266648 - Rollout of
ciStageUpdated, closed over cardinality concerns: #581569 (closed) - Burst of 668 triggers on one stage: https://gitlab.com/gitlab-org/gitlab/-/work_items/630854#note_4059560153
