Defer MR diffs cache write to a worker on creation

What

Defers the merge request diffs highlight/stats cache write in MergeRequests::AfterCreateService to a new Sidekiq worker (MergeRequests::WriteDiffsCacheWorker), behind the async_write_diffs_cache_on_mr_create feature flag (gitlab_com_derisk, default off).

Why

NewMergeRequestWorkerMergeRequests::AfterCreateService is a high-urgency, CPU-bound worker expected to complete within 10s. Per-method duration logging (the now-removed log_merge_request_after_create_duration flag, rollout #579080 (closed)) showed the synchronous merge_request.diffs(include_stats: false).write_cache call reaching ~11s at P99 on large merge requests — it's syntax highlighting (O(files × lines)) plus per-file Redis writes.

The cache is display-only and not on the readiness path, so it can be warmed asynchronously, off the creation critical path.

Safety

  • Display-only cache — nothing in MR preparation, mergeability, pipelines, approvals, or webhooks reads it.
  • On-demand fallback — if the cache is cold when a diff is viewed, the render path highlights live and writes it (write_if_empty). A cold cache means a slightly slower first render, never missing/incorrect data.
  • Content-addressed key (derived from the immutable merge request diff) ⇒ no staleness or races; a concurrent on-view write and the worker write produce identical data (per-field HSET with deterministic, idempotent values).
  • Worker is idempotent! (write_if_empty no-ops when warm), data_consistency :sticky (reads the replica when it is caught up, otherwise the primary — so it always warms from the just-created diff), urgency :low, worker_resource_boundary :cpu.
  • Gated by async_write_diffs_cache_on_mr_create (default off) ⇒ instant rollback, and the new worker is not scheduled until it is deployed everywhere.

Test plan

  • New spec/workers/merge_requests/write_diffs_cache_worker_spec.rb (includes the an idempotent worker shared example).
  • spec/services/merge_requests/after_create_service_spec.rb covers both flag states (async enqueue and synchronous fallback).

Follow-ups before enabling

  • Create the feature flag rollout issue and fill rollout_issue_url in the flag definition — #604642.

Related to #417973 (closed)

Edited by Marc Shaw

Merge request reports

Loading