Introduce downstream pipeline trigger rate limit

What does this MR do and why?

This MR introduces a new rate limit and logging for downstream pipelines triggered by Ci::Bridge jobs.

This limit is not enforced unless we enable the Feature Flag: ci_rate_limit_downstream_pipelines. (Roll out issue: gitlab-com/gl-infra/production#17471 (closed)).

For now this limit is hardcoded. We will set a dedicated time period (1 week+) where the FF will remain off so we can observe the logs. This data will help inform us of if/when we should convert it into an application setting and what its value should be for .com. The new setting would be named downstream_pipeline_trigger_limit_per_project_user_sha or similar.

Currently, we already enforce a rate limit (pipeline_limit_per_project_user_sha) on creating multi-project pipelines. However, this new limit is specific to triggering both child and multi-project downstream pipelines.

This does mean the new limit may have some overlap with the existing limits; it depends on which is reached first. The screenshots below show that the visual difference in the UI is minimal.

MR acceptance checklist

Please evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Screenshots or screen recordings

1. Currently, if the existing pipeline_limit_per_project_user_sha is exceeded first.

Pipeline view	Error message

2. If the new limit is exceeded first.

Pipeline view	Error message

How to set up and validate locally

1. Enable the feature flag:

Feature.enable(:ci_rate_limit_downstream_pipelines)

2. Update the limit to 2 in your local file app/services/ci/trigger_downstream_pipeline_service.rb so that we don't have to trigger a lot of downstream pipelines.

DOWNSTREAM_PIPELINE_TRIGGER_LIMIT_PER_PROJECT_USER_SHA = 2

3. Create two projects with the following configuration.

Project 1:

• Create file config1.yml with:

job:
    script: echo

• Update .gitlab-ci.yml file with:

trigger-child-A:
    trigger:
        include: config1.yml

trigger-project-2-main-A:
    trigger:
        project: group-a/project-2

trigger-child-B:
    trigger:
        include: config1.yml

trigger-project-2-main-B:
    trigger:
        project: group-a/project-2

Project 2:

• Update .gitlab-ci.yml file with:

job:
    script: echo

3. Run the pipeline and observe that only 2 trigger jobs succeed. Also observe that the limit applies to both child and multi-project pipelines.