Fork API returns 409 error despite successful project creation (race condition)

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

Summary

The POST /api/v4/projects/:id/fork endpoint sometimes returns a 409 Conflict error with the message {message: [Project namespace name has already been taken, Name has already been taken, Path has already been taken]}, even though the fork was successfully created at the exact time the API returned the error.

This is a race condition in the project creation transaction that causes false-negative responses, making the API unreliable for automated workflows.

Steps to reproduce

  1. Use the fork API endpoint: POST /api/v4/projects/:id/fork

  2. Observe that occasionally (intermittently) the API returns:

    409 Conflict
{message: [Project namespace name has already been taken, Name has already been taken, Path has already been taken]}

  3. Check the target namespace - the forked project exists and was created at the exact timestamp of the API call

Current behavior

  • API returns 409 Conflict error
  • Error message claims the path/name is already taken
  • However, the project was successfully created in the database
  • This happens intermittently, suggesting a race condition

Expected behavior

  • If the fork is created successfully, return 201 Created with the project details
  • If the fork fails due to a duplicate path, return 409 Conflict and do not create the project
  • The API response should accurately reflect the actual outcome

Environment

  • GitLab instance: Self-hosted (gitlab.example.com)
  • API endpoint: POST https://gitlab.example.com/api/v4/projects/244795/fork
  • Frequency: Intermittent (race condition)

Root Cause Analysis

Based on investigation of the GitLab codebase, this is a time-of-check to time-of-use (TOCTOU) race condition in the project creation transaction.

The Critical Code Path

In app/services/projects/create_service.rb:251-276:

ApplicationRecord.transaction do
  @project.build_or_assign_import_data(data: @import_data[:data], credentials: @import_data[:credentials]) if @import_data

  # Avoid project callbacks being triggered multiple times by saving the parent first.
  # See https://github.com/rails/rails/issues/41701.
  Namespaces::ProjectNamespace.create_from_project!(@project) if @project.valid?  # Line 260

  if @project.saved?
    Integration.create_from_default_integrations(@project, :project_id)
    # ...
  end
end

In app/models/namespaces/project_namespace.rb:42-50:

def self.create_from_project!(project)
  return unless project.new_record?
  return unless project.namespace

  proj_namespace = project.project_namespace || project.build_project_namespace
  project.project_namespace.sync_attributes_from_project(project)
  proj_namespace.save!  # Validation runs AGAIN here - Line 48
  proj_namespace
end

In app/models/route.rb:15:

validates :path, uniqueness: { case_sensitive: false }

What's Happening

  1. First validation (@project.valid? at line 260): Checks uniqueness of name/path - passes because no duplicate exists yet
  2. Race window: Another concurrent fork request can slip in and create the same path
  3. Second validation (inside proj_namespace.save! at line 48): The Route model validates path uniqueness globally - fails because the concurrent request created the route
  4. Database trigger side effect: Due to the database trigger that synchronizes projects and namespaces tables (see app/models/project.rb:199-200), the project record may already be partially persisted:
# Sync deletion via DB Trigger to ensure we do not have
# a project without a project_namespace (or vice-versa)
belongs_to :project_namespace, autosave: true, class_name: 'Namespaces::ProjectNamespace', foreign_key: 'project_namespace_id', inverse_of: :project

Why This Creates the Observed Behavior

  • The project gets created successfully by the first/winning request's database trigger
  • The second/losing request hits the duplicate validation during proj_namespace.save!
  • The validation errors propagate back to the API response as a 409 error
  • But the project already exists in the database from the earlier transaction

This is a classic ActiveRecord race condition: the uniqueness validation performs a separate SELECT query before the INSERT, leaving a window for concurrent requests to both pass validation before either commits.

Proposed Solutions

  1. Database-level locking: Use SELECT FOR UPDATE on the parent namespace before creating the fork
  2. Database constraints only: Rely solely on database constraints instead of ActiveRecord validations for uniqueness, and handle constraint violations gracefully
  3. Advisory locks: Use PostgreSQL advisory locks to serialize fork creation for the same namespace/path combination
  4. Optimistic locking with retry: Implement retry logic that detects and handles the race condition gracefully
  5. Idempotency check: Before returning 409, verify whether the project actually exists and was created by the current request

Impact

  • API reliability: Clients cannot trust the HTTP status code
  • Automation failures: CI/CD pipelines and automation tools fail unnecessarily
  • User confusion: Users see errors but the operation succeeded
  • Retry storms: Clients may retry the operation, potentially creating duplicate projects or wasting resources

Related Issues

  • #214127 (closed) - Similar symptoms but was user error (incorrect API parameters)
  • #535568 (closed) - Similar 409 errors with Terraform provider, closed as support request

Perfect! This is ready to submit to the GitLab issue tracker.

Edited by 🤖 GitLab Bot 🤖