Skip to content

Visualize debug log for pipeline creation

Release notes

Problem to solve

The GitLab support team is struggling to quickly identify the root cause of pipeline or job failures, which often involve changes in pipeline definitions, environments, or the codebase. The complexity is increased by the use of pipeline templates stored in dedicated repositories with intricate referencing, leading to pipelines with thousands of lines of configuration, even if the main .gitlab-ci.yml is simple.

When collaborators modify template repositories, pipelines inherit the new changes, making it hard to track what has changed between pipeline runs. The team needs the ability to debug what has happened during the pipeline creation.

Intended users

User experience goal

When visualizing any specific pipeline run, the user should be able to navigate to a dedicated tab or section within a given pipeline run, where they'll have easy access to each CI file used to execute the pipeline, and their respective git ref used at the time the pipeline was run.

This feature aims to simplify the understanding of the CI configuration associated with each pipeline, improving the overall user experience.

Implementation idea

Introduce the ability to profile pipeline creation on-demand and save the log as pipeline artifact:

  • the profiling is disabled by default for obvious performance reasons. It can be enabled via the UI Run pipeline form by checking a check box Collect debug information of pipeline creation . Maybe allow only maintainers to set that and eventually inspect the log. Maybe allow that only via the UI initially to avoid that automations using REST API have it always enabled.
  • When the debug mode is enabled we always setup the pipeline logger (where we instrument all the steps) or a new dedicated logger and export the results.
  • We can provide a summary: where is most of the time spent? Ratio of included vs defined jobs, etc.
  • The debug dump file (could be a JSON export) is then uploaded as pipeline artifact.
  • Maintainers can read/download the debug dump file from the UI.
  • The frontend could parse the file to provide visual inspection integrated in the GitLab UI.
  • Include API support where this same information can be fetched and parsed, for automation purposes.

Useful info to include:

  • merged yaml
  • ref/SHA or each included template
  • debug info about workflow:rules and each job:rules (what condition passed/failed?, what variables were used? don't leak values!)
  • what Scan Execution Policy or Pipeline Execution Policy was enforced (include ref/SHA of each + cascade debug log into creation of PEP pipelines)
  • where is most of the time spent? Are yweou fetching a remote template from a server with high latency? Are we evaluating the same conditions hundreds of times?
  • Ratio of included vs defined jobs - are you defining many jobs but actually running 10% of them? If so, you may benefit from using include:rules upstream and restructuring your YAML.

Security and Reliability

  • Ensure no sensitive information are leaked in the log such as masked variables.
  • Allow only maintainer+ to enable debug log.
  • Rate-limit the operation to avoid users abusing it, as creating a debug log will make pipeline creation slower.
  • Set default/hard-coded expiration time for the debug log artifact so they don't linger forever.

Documentation

Documentation will be provided to guide users on accessing and utilizing the new CI code visualization feature.

Availability & Testing

  • TBD

Available Tier

  • TBD

Feature Usage Metrics

  • TBD

What does success look like, and how can we measure that?

  • Decrease in support tickets for troubleshooting high complexity pipeline failures

Is this a cross-stage feature?

  • TBD

What is the competitive advantage or differentiation for this feature?

  • This feature adds support for high complexity CI/CD environments, which is the beating heart of GitLab.

Links / references

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

Edited by 🤖 GitLab Bot 🤖