Skip pipelines for non-default branches on first mirror sync

Kerri Millerrequested to merge
598486-pull-mirror-first-sync-skip-pipelines into master

What does this MR do and why?

Fixes an issue where setting up a pull mirror on a repository with many branches and tags triggers thousands of pipelines, one for each ref. This caused an incident on GitLab Dedicated (see related incident).

Problem

When a pull mirror is first set up, GitLab fetches all refs from the remote and creates local branches/tags. Each branch/tag creation triggers a post-receive hook, which in turn triggers a CI pipeline. For repositories with hundreds or thousands of refs, this creates a "pipeline explosion" that can overwhelm the system.

Solution

On first sync only, we now:

  1. Create the default branch normally (allowing its pipeline to run)
  2. Create all other branches with skip_ci: true (preventing pipelines)
  3. Create all tags with create_pipelines: false (preventing pipelines)

Subsequent (delta) syncs continue to work normally - all changed refs trigger pipelines as expected.

This matches user expectations: when enabling "Trigger pipelines for mirror updates", users expect to see a pipeline for the default branch, not hundreds of pipelines for every historical branch.

Multi-release deployment

Per Sidekiq compatibility guidelines, adding a new worker argument requires a multi-release deployment:

Release Changes
19.3 (this MR) Worker: Add skip_ci = false parameter (backwards compatible)
19.4 (#607902) Caller: Start passing skip_ci argument

Feature flag

This change is behind the skip_pipelines_for_mirror_first_sync_non_default_branches feature flag (disabled by default).

Closes #598486

How to set up and validate locally

  1. Enable the feature flag:

    Feature.enable(:skip_pipelines_for_mirror_first_sync_non_default_branches)

  2. Create a new project with pull mirroring enabled:

    • Settings > Repository > Mirroring repositories
    • Add a mirror URL pointing to a repo with multiple branches
    • Enable "Trigger pipelines for mirror updates"

  3. Trigger the first mirror sync

  4. Verify:

    • Only ONE pipeline is created (for the default branch)
    • Other branches are created but have no pipelines
    • Tags are created but have no pipelines

  5. Trigger a second sync (delta sync) by pushing to the remote

  6. Verify:

    • Changed refs trigger pipelines normally

MR acceptance checklist

  • Tests added for new functionality
  • Feature flag added
  • Rollout issue created
  • Multi-release deployment: follow-up issue created

Related issues

Edited by Kerri Miller

Merge request reports