Skip to content

Make trace-pipeline job run before pipeline_and_job_duration job

Reuben Pereira requested to merge rp/tracing-job-runs-first into master

What does this MR do and why?

Describe in detail what your merge request does and why.

If the trace-pipeline and pipeline_and_job_duration jobs run at the same time, the pipeline_and_job_duration job might finish first which results in the traces generated by trace-pipeline containing the pipeline_and_job_duration job as well.

Since the pipeline_and_job_duration job is executed many hours after (because it is triggered by a manual job) the rest of the pipeline, this can skew the trace of the pipeline. Here is an example trace that is skewed by the metrics:pipeline_and_job_duration job: https://observe.gitlab.com/v1/jaeger/1112072/trace/0ff0bca252ad336472acfe15b0b6f866 (scroll right to the bottom).

By running the trace job first, we avoid this problem.

The duration metrics recorded by the pipeline_and_job_duration job can also be skewed because of the same problem. To solve this, the trace-pipeline job has been explicitly ignored when recording pipeline durations.

Author Check-list

  • Has documentation been updated?
Edited by Reuben Pereira

Merge request reports