Skip to content

Handle Duplication of Resources on Migration

When migrating projects using the new :bulk_import_projects feature flag, resources can be duplicated due to race conditions.

See the related discussion for examples, but the pipeline will run multiple times for the same resource

The following discussion from !74100 (merged) should be addressed:

  • @acook.gitlab started a discussion: (+14 comments)

    @georgekoltsov I've investigated this with @Alexand and this change is due to a race condition.

    1. PipelineSchedulesPipeline starts importing a schedule
    2. Another worker runs, sees it's started then starts it again
    3. You end up with multiple copies of the same pipeline schedule imported

    When the worker sees a pipeline is already started for this, should it still try to run? This seems counterintuitive to me

    There's an explicit test looking for this functionality though added on 9/28 by you so I think we might need to discuss what to do here