Bound the release evidence pipeline lookup

What does this MR do and why?

Context

When we collect release evidence, we look up the last pipeline for the release's commit to attach its test reports. On a commit with many pipelines that lookup can run until the statement timeout, and it runs on every release create and every hourly evidence run. This MR is part 3 of the plan in https://gitlab.com/gitlab-org/gitlab/-/work_items/638257#note_4197891367, which has the figures and the approach; parts 1 and 2 have their own MRs.

This MR

  • Release creates and the hourly evidence run look up the pipeline under the fast read statement timeout on the CI database. When the lookup times out, we log it and collect the evidence without a pipeline, so the release keeps its evidence but without test reports. A create no longer fails, and the hourly run no longer skips the release, on a slow lookup.
  • The evidence API (POST /projects/:id/releases/:tag_name/evidence) no longer runs the lookup. It passed the pipeline to the worker as an object, which Sidekiq serializes to a string the worker never finds, so evidence collected through the API is unchanged.
  • With the flag off, both paths behave as on master.
  • The Changelog entry comes with the flag-removal MR.

Feature Flag: release_evidence_bounded_pipeline_lookup

References

Database queries

See MR comments.

How to set up and validate locally

  1. In a Rails console, pick a release whose commit has a pipeline, and enable the flag:
    release = Release.joins(:project).find { |r| r.project.ci_pipelines.for_sha(r.sha).exists? }
    Feature.enable(:release_evidence_bounded_pipeline_lookup)
    Releases::EvidencePipelineFinder.new(release.project, tag: release.tag).execute
    It returns the same pipeline as with the flag off, and log/development.log shows SET LOCAL statement_timeout = 4500 on the ci connection before the p_ci_pipelines query.
  2. Collect evidence through the API:
    curl --request POST --header "PRIVATE-TOKEN: <your_access_token>" \
      "http://gdk.test:3000/api/v4/projects/<project_id>/releases/<tag_name>/evidence"
    It returns 202, a new evidence appears for the release, and the request runs no p_ci_pipelines query.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Leaminn Ma

Merge request reports

Loading
Loading