Echo Job Router last_update for long-poll coalescing

What does this MR do?

Echo Job Router last_update for long-poll coalescing

The Job Router path proxies job requests through Relay (KAS) over gRPC, so responses never pass through the network client's HTTP layer that normally records the X-GitLab-Last-Update token. Without that token the runner cannot advertise the queue version it last saw, so Workhorse answers immediately instead of coalescing the poll.

Read the token that Relay relays back as gRPC metadata and store it in the per-credentials network client via SetLastUpdate, so the next PrepareJobRequest echoes it as last_update.

Why was this MR needed?

See above.

What's the best way to test this MR?

  1. Configure the following in gdk.yml

    workhorse:
      ci_long_polling_seconds: 30
  2. gdk reconfigure && gdk restart

  3. cd "gitlab" && git checkout vtak/job_router_longpoll_coalescing && make -C workhorse && gdk restart gitlab-workhorse && gdk tail gitlab-workhorse

  4. Enable Rails feature flags: job_router using the follwing Rails command - Feature.enable(:job_router)

  5. Start GitLab Relay(KAS).

  6. Setup a runner with FF_USE_JOB_ROUTER = true environment variable feature flag and start GitLab Runner.

  7. Search the Worhorse logs for "/api/v4/internal/ci/job_router/jobs/request" and verify the the following.

    1. First entry should have read_bytes as 1588 and all the following entries as 1638. This confirms all the following entries have the last_update passed in the header from GitLab Runner.

    2. All entries have duration_ms almost equal to 30000 ms. This means that long poll is being respected.

NOTE: See #39383 (comment 3656946312) for the testing already done.

What are the relevant issue numbers?

Edited by Vishal Tak

Merge request reports

Loading