Skip to content

Deduplicate token triggered pipelines

Alex Kalderimis requested to merge 352158-dedup-api-trigger-pipelines into master

What does this MR do and why?

This attempts to address #352158 by limiting CI pipeline creation when the REST API project pipeline trigger endpoint is called.

We prevent creating further pipelines if 50 running pipelines or more already exist for the same SHA/ref.

The abuse this is designed to prevent is mostly accidental. We had a customer incident where a very large number (several thousand) of pipelines were triggered via this API on the same ref for the same project, to the point where queuing and service degradation was observed. This MR is directly aimed at preventing this specific scenario, since we are still a little unsure how so many pipelines were created (a webhook loop? a bug in their code? The explanation from the customer doesn't seem to match the scale of API calls we saw).

The aim of this MR is not deliberate abuse prevention. Free accounts already have limits on pipelines. This is a safety catch.

Related issues

See #352232, which aims to generalize this approach to rate limiting pipeline creation by SHA/REF

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #352158

Edited by Alex Kalderimis

Merge request reports