Debian Package Registry: Race condition

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Hello, i am running gitlab v17.10.3-ee and i use curl to upload 50+ .deb packages. Like this:

      for file in *.deb; do
        sleep 3 #half of packages is not uploaded without this (despite returning HTTP 201)
        curl --fail --no-progress-meter -w "\tHTTP:%{response_code}:\t$file\n" \
             --request PUT --user "gitlab-ci-token:$CI_JOB_TOKEN" \
             --upload-file "$file" \
             "${CI_API_V4_URL}/projects/${CI_PROJECT_ID}/packages/debian/${file}?distribution=${DISTRIB}&component=main"
      done

Problem is that when i do not sleep for couple seconds between uploads, some of the files are not correctly saved in the registry (seems to be random).

I don't think relying on sleep is correct and there is probably some kind of underlying race condition taht needs to be fixed for this to work reliably.

Maybe one of the factors might be that i have multiple .deb files in each package (different architectures, debug packages, etc...). This is screenshot of correctly uploaded package with all files demonstrating it:

(maybe when second file is uploaded to the package that was very recently created it's not correctly saved, because some background job is still running even after HTTP 201 was returned)

I've noticed that API calls on debian registry are not synchronous. (eg. it takes some time after creating new debian_distribution before InRelease becomes available). So i guess it's probably similar when uploading packages.

Also i wonder if this would affect dput uploads as well.