Add status, timing, and merge request data to the user pipelines API

What does this MR do and why?

The experimental personal-homepage pipelines widget needs richer pipeline data than the instance-level GET /pipelines endpoint currently returns. This MR adds three groups of fields to that endpoint:

  1. detailed_status — the server-computed status object (icon, text, label, group, tooltip, has_details, details_path, favicon), serialized with the existing shared DetailedStatusEntity. This lets clients render statuses like "passed with warnings" that the raw status string can't distinguish.
  2. started_at, finished_at, duration, queued_duration — the same timing fields already exposed by the single-pipeline endpoint.
  3. merge_request (iid, title, web_url) — included only for merge request pipelines whose merge request the user can read (checked with Ability.allowed?(:read_merge_request)). Serialized with a new slim API::Entities::Ci::PipelineMergeRequest entity that deliberately excludes the description to keep list payloads small.

Implementation: a new entity API::Entities::Ci::UserPipeline subclasses PipelineBasicWithProject, and lib/api/ci/user_pipelines.rb presents it with current_user. A new model scope Ci::Pipeline.with_user_pipelines_api_associations extends the existing preloads with the merge request, its author, and its target project routes.

This MR is stacked on top of !252943 (merged) and targets that branch, so the diff here shows only the new commits.

How to set up and validate locally

  1. Run GDK.
  2. Trigger a few pipelines as your user, including at least one merge request pipeline.
  3. Run:
curl --request GET --header "PRIVATE-TOKEN: <your_access_token>" --url "http://gdk.test:3000/api/v4/pipelines?per_page=5"
  1. Confirm the response includes detailed_status, the timing fields, and merge_request (for merge request pipelines only).

Performance

detailed_status needs a failed-but-allowed-jobs check for successful pipelines ("passed with warnings"). That lookup is BatchLoader-based, and the endpoint registers all pipelines with the BatchLoader before serialization, so it collapses into a single grouped query per page instead of one per pipeline.

The merge_request exposure preloads the merge request, its author (the read policy checks whether the author is banned, which otherwise loaded each author individually), and its target project with routes — a fixed number of queries per page, regardless of how many pipelines are returned.

The N+1 QueryRecorder spec was extended to include merge request pipelines across multiple projects and passes: 25 examples, 0 failures.

Backwards compatibility

  • Purely additive response fields; nothing removed, renamed, or retyped — non-breaking under REST v4 guidelines.
  • merge_request is conditional: omitted for non-merge-request pipelines and for merge requests the user cannot read (for example, in fork constellations or when the merge requests feature is disabled).
  • The project-scoped GET /projects/:id/pipelines endpoint and its entities are untouched.
  • Docs (doc/api/pipelines.md) and the OpenAPI definition were updated.

Database

The new scope Ci::Pipeline.with_user_pipelines_api_associations only adds preloads on top of the existing with_api_entity_associations; the paginated base query is unchanged from the endpoint's introduction in !250635 (merged) (plans in that MR's description). The queries below were captured from the endpoint code path and analyzed with EXPLAIN ANALYZE on Database Lab clones.

Warnings count (BatchLoader, one query per page)

Runs once per page for detailed_status (the "passed with warnings" check), batched over all pipelines on the page:

SELECT COUNT(*) AS count_all, p_ci_builds.commit_id, p_ci_builds.partition_id
FROM p_ci_builds
WHERE (p_ci_builds.commit_id = ? AND p_ci_builds.partition_id = ? OR ...) -- one pair per pipeline
  AND (p_ci_builds.retried = FALSE OR p_ci_builds.retried IS NULL)
  AND p_ci_builds.allow_failure = TRUE
  AND p_ci_builds.status IN ('failed', 'canceled')
GROUP BY p_ci_builds.commit_id, p_ci_builds.partition_id

Plan on the CI clone (5 pipelines): partition-pruned bitmap index scans on ci_builds_<partition>_commit_id_status_type_idx. Total 30.4 ms (planning 8.1 ms, execution 22.3 ms), 35 buffer hits + 8 reads.

merge_requests preload

SELECT merge_requests.* FROM merge_requests WHERE merge_requests.id IN (...)

Index Scan using merge_requests_pkey. 16.3 ms (planning 6.1 ms, execution 10.2 ms), 2 buffer hits + 7 reads.

projects preload (merge request target projects)

SELECT projects.* FROM projects WHERE projects.id IN (...)

Primary-key index scan. 24.2 ms (planning 4.7 ms, execution 19.6 ms), 7 buffer hits + 15 reads.

routes preload

SELECT routes.* FROM routes WHERE routes.source_type = 'Project' AND routes.source_id IN (...)

Index scan on (source_type, source_id). 29.0 ms (planning 1.7 ms, execution 27.3 ms), 5 buffer hits + 15 reads.

The remaining preloads (users for merge request authors, namespaces, project_features) are primary-key or unique-index IN lookups of the same shape and cost as the queries above.

CI partition-pruning check

The check flags fingerprint 955968b0f17eca7c (Ci::Pipeline.latest_completed_or_manual_pipeline_ids_per_source). That query predates this MR and comes from the scan result policies code path, which the new merge request pipeline specs happen to exercise. It is registered in scripts/database/query_analyzers.yml todos: with the fix tracked in #627306.

References

Edited by Marcel van Remmerden

Merge request reports

Loading
Loading