Return 503 from the compare API after a Gitaly diff timeout

What does this MR do and why?

When computing the diff for GET /projects/:id/repository/compare hits the Gitaly CommitDiff deadline (30 seconds on GitLab.com), the request fails with a 500 and nothing is cached, so a caller that retries pays the full deadline again on every attempt.

After a timeout, the endpoint now records the comparison (project, resolved SHAs, and straight) in Rails.cache for ten minutes and answers repeat requests for the same range with 503 Service Unavailable and a Retry-After header, without calling Gitaly. The first failing request also returns 503 with Retry-After, so callers see one consistent status. A different range on the same project, or the same range in the other comparison mode, is unaffected.

This is behind the compare_api_diff_timeout_cache feature flag, off by default. With the flag off, behaviour is unchanged.

Precedent: the negative cache follows Gitlab::Git::CommitStats#gitaly_commit_stats (commit_stats_failure_handling), and the 503 follows the archive, tags and files endpoints, which already rescue Gitlab::Git::CommandError with service_unavailable!.

Rollback: disable compare_api_diff_timeout_cache.

References

Detailed context for reviewers and AI agents

Shape

  • compare_range resolves the SHAs once in the endpoint; the response cache key (from !268437 (merged)) and the timeout marker both use it. The anonymous commit cap is now in !268855 (merged), stacked separately.
  • with_compare_timeout_cache wraps cache_action. It checks the marker before yielding, and on Gitlab::Git::CommandTimedOut writes the marker and renders the 503. The begin/rescue is deliberately inside the flag check so a timeout with the flag off propagates exactly as before.
  • COMPARE_TIMED_OUT_EXPIRY = 10.minutes, matching Gitlab::Git::CommitStats::TIMED_OUT_EXPIRY.
  • desc now declares failure responses (400, 403, 404, 503); doc/api/openapi/openapi_v3.yaml is regenerated.

Choices to confirm

  • 503 rather than a degraded 200. CommitDetail#stats degrades to nil on a timeout, but diffs are this endpoint's primary purpose, and an empty diffs array would read as "no changes". Changing a 500 is not a breaking change under the REST API guidelines.
  • Marker keyed on resolved SHAs. A retry that spells the ref differently still hits the marker. A legitimate caller on a different path through the same range is also blocked for the TTL; ten minutes seems an acceptable trade.
  • CommandTimedOut only. Other CommandErrors keep returning 500, as today. Mapping them to 503 as the sibling endpoints do would be a reasonable follow-up but is out of scope here.

Docs

doc/api/repositories.md: history entry, flag note, and a paragraph on the 503.

How to set up and validate locally

  1. In the Rails console, enable the flag and make CommitDiff time out:

    Feature.enable(:compare_api_diff_timeout_cache)
    Gitlab::Git::Diff.singleton_class.prepend(Module.new { def between(*) = raise(Gitlab::Git::CommandTimedOut, 'deadline exceeded') })
  2. Request a comparison:

    curl --include --header "PRIVATE-TOKEN: <token>" \
      --url "http://gdk.test:3000/api/v4/projects/<project id>/repository/compare?from=<base>&to=<head>"

    The response is 503 with Retry-After: 600.

  3. Restart the console (dropping the override) and repeat the request. It is still 503, and no Gitaly call is made.

  4. Request a different range on the same project. It succeeds.

  5. Disable the flag and repeat step 2 with the override. The response is 500, as before.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Gavin Hinfey

Merge request reports

Loading
Loading