Add filtering parameters (pipeline_id) to webhook events API for disaster recovery scenarios

Release notes

Problem to solve

The Get webhook events API lacks filtering parameters, making it impractical to retrieve webhook events for specific pipelines during disaster recovery scenarios.

Current behavior:

  • API returns all webhook events from the past 7 days with no filtering options
  • Users must download and parse thousands of events to find events for specific pipeline runs
  • No way to filter by pipeline_id, date range, or event type

Problem scenario: When an integration platform experiences an outage:

  1. User identifies which pipelines ran during the outage (via List project pipelines API with created_after/created_before)
  2. User needs webhook events for those specific pipeline IDs to reprocess via Resend webhook API
  3. User must download entire 7-day webhook event history (potentially thousands of events)
  4. User must parse every event to extract pipeline_id and filter client-side

For organizations with high webhook volume, this performance bottleneck makes disaster recovery impractical and blocks industrialization of GitLab CI integrations.

What was initially requested: A correlation ID to group events from the same pipeline run.

What's actually needed: Server-side filtering on the webhook events API to retrieve only relevant events without downloading the entire history.

Intended users

Administrators of External Services Integrated with GitLab (e.g. OpenText Octane/ValueEdge)

Use Case

Disaster recovery for an external service integrated with GitLab.

When an external service goes down, customers want to:

  1. Query GitLab for missed webhook events
  2. Identify which events belong to the same pipeline run
  3. Resend only specific batches of events (not all events)
  4. Avoid processing large data payloads unnecessarily

1. On-Demand Recovery

  • Reprocess events for a single pipeline: filter by pipeline_id=12345
  • Reprocess events for multiple specific pipelines: filter by pipeline_id[]=12345&pipeline_id[]=67890

2. Outage Recovery Current workflow during DR:

  1. Use GET /projects/:id/pipelines with created_after and created_before to identify pipelines that ran during outage
  2. Extract list of pipeline IDs from response
  3. Query webhook events for those specific pipeline IDs
  4. Reprocess only the relevant events

User experience goal

The users should be able to resend only specific webhook events based on the pipeline run/re-run, thus avoiding processing large data payloads in case of disaster recovery scenarios, where the user would like to re-inject the missed/unprocessed webhook events due to user platform/product downtime.

Proposal

Add filtering query parameters to the GET /projects/:id/hooks/:hook_id/events API endpoint:

Required parameter:

  • pipeline_id - Filter events by single pipeline ID or array of pipeline IDs

Optional parameters (for future consideration):

  • created_after - Filter events created after timestamp
  • created_before - Filter events created before timestamp
  • event_type - Filter by webhook event type (pipeline, job, etc.)

Example API calls:

`# Single pipeline GET /api/v4/projects/123/hooks/456/events?pipeline_id=12345

Multiple pipelines

GET /api/v4/projects/123/hooks/456/events?pipeline_id[]=12345&pipeline_id[]=67890`

Further details

Permissions and Security

  • Add expected impact to Maintainer (40) members
  • Add expected impact to Owner (50) members

Documentation

Availability & Testing

Available Tier

  • Free
  • Premium
  • Ultimate

Feature Usage Metrics

What does success look like, and how can we measure that?

To have this feature implemented.

What is the type of buyer?

Is this a cross-stage feature?

Yes

What is the competitive advantage or differentiation for this feature?

Links / references

Edited by 🤖 GitLab Bot 🤖