View Pipeline and Job Provenance
# Problem to Solve
Customers are interested in seeing who has performed an action (e.g. cancel, retry) on a pipeline or a job to help with debugging and accountability.
Some events can be inferred through API or UI while for some events it's unclear where and how to get the information (e.g. cancellation, retry)
## Audit need
Organisations need to track who triggers pipelines, on which projects, and what jobs are executed for security compliance and audit purposes. Currently, GitLab lacks comprehensive pipeline execution auditing in its formal audit event systems. While [existing audit features capture some CI/CD actions](https://docs.gitlab.com/user/compliance/audit_event_types/#continuous-integration-1), they don't provide complete pipeline execution history, especially for tracking potentially malicious users who might execute destructive jobs and then delete pipelines to hide their activities. An immutable record of pipeline executions that persists even after pipeline deletion is needed. The current approach of manually querying the GitLab API for pipeline information is inefficient and impractical at scale.
##
# Proposal
We propose adding a new comprehensive Pipeline Execution Audit Event that would capture the following data points:
1. **Pipeline Trigger Information**:
* User who triggered the pipeline (username, user ID, name)
* Trigger method (manual, API, webhook, scheduled, merge request, etc.)
* Timestamp of pipeline initiation
2. **Pipeline Context**:
* Project ID and name
* Repository reference (branch/tag/commit)
* Pipeline ID and configuration source
3. **Job Execution Details**:
* List of jobs executed in the pipeline
* Job status (success, failed, canceled)
* Job configuration (runner type, variables used - excluding secrets)
* Job start and end timestamps
4. **Pipeline Lifecycle Events**:
* Pipeline creation
* Pipeline status changes
* Pipeline deletion (with retention of the audit record)
These events should be generated at key points in the pipeline lifecycle, particularly:
* When a pipeline is created/triggered
* When pipeline status changes
* When a pipeline or its jobs are deleted
### Streaming-only event or normal event?
We recommend implementing this as a **normal event** with the option to configure it as streaming-only for large-scale deployments.
For smaller instances, storing this information in GitLab's database provides integrated access and native UI for audit review. However, for larger installations, the volume of audit data could be substantial.
Therefore, we suggest:
1. Implementing this as a normal database-stored event by default for smaller installations
2. Providing configuration options to make it streaming-only for large installations
3. Adding retention policies specifically for pipeline audit events to prevent database bloat
This approach balances accessibility of audit information for smaller teams while providing scalability options for larger enterprises.
# Pricing Tier - Proposed
Premium/Ultimate customers: All pipeline events available in the Audit log ( https://gitlab.com/gitlab-org/gitlab/-/issues/377608)
All customers: Who/when information about critical events (TBD) available on Pipeline/Job pages.
epic