Skip to content

Mitigate the race condition when creating a new npm package

🌱 Context

While performing an investigation of slow SQL queries it was revealed that there duplicate npm packages in the database

What does this MR do and why?

Obtain a lease when create a new npm package in order to exclude the chance of creating duplicate packages.

🔬 How to set up and validate locally

  1. Create a rake task lib/tasks/packages/race_condition.rake that's going to simulate a parallel creation of npm packages using Thread.

    # frozen_string_literal: true
    
    desc 'GitLab | Packages | Race condition'
    namespace :gitlab do
      namespace :packages do
        task race_condition: :environment do
          # Similar setup to what we have in the tests
          user = User.first
          project = Project.find(1)
          params = JSON.parse(
            File.read('spec/fixtures/packages/npm/payload.json').gsub('@root/npm-test', 'my-package')
          ).with_indifferent_access
    
          wait_for_it = true
    
          threads = Array.new(10) do |i|
            Thread.new do
              true while wait_for_it
    
              ::Packages::Npm::CreatePackageService.new(project, user, params).execute
            end
          end
          wait_for_it = false
          threads.each(&:join)
        end
      end
    end
  2. Execute the rake task from the previous step.

    $ rake gitlab:packages:race_condition
  3. In rails console verify amount of created npm packages. It should be 1

    Packages::Package.where(name: 'my-package').count
  4. After the verification is done, it's time to delete the rake task.

    $ rm -rf lib/tasks/packages

🏁 MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #390097 (closed)

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports