Packages uploads fails with: object storage enabled, direct upload disabled, background upload enabled
Problem to solve
This is a follow-up of !26387 (comment 298710168) (See case 6.)
When:
- Object Storage enabled
- Direct Upload disabled
- Background Upload enabled
in the packages section of the config we get:
Sidekiq::Worker::EnqueueFromTransactionError (`ObjectStorage::BackgroundMoveWorker.perform_async` cannot be called inside a transaction as this can lead to
race conditions when the worker runs before the transaction is committed and
tries to access a model that has not been saved yet.
Use an `after_commit` hook, or include `AfterCommitQueue` and use a `run_after_commit` block instead.
):
/Users/david/projects/gitlab-development-kit/gitlab/config/initializers/forbid_sidekiq_in_transactions.rb:23:in `block (2 levels) in <module:NoEnqueueingFromTransactions>'
/Users/david/projects/gitlab-development-kit/gitlab/app/uploaders/object_storage.rb:270:in `schedule_background_upload'
Reach
- This is happening on
NuGet and NPM packagesall package types.
Suggested Fix
Use include ObjectStorage::BackgroundMove
in Packages::PackageFile
in place of having after :store, :schedule_background_upload
in Packages::PackageFileUploader
(according to https://docs.gitlab.com/ee/development/file_storage.html#using-objectstorageextensionrecordsuploads).
It fixes the error above but it messes with the NuGet background worker. Basically, the background move worker and the NuGet background worker try both to access the file and that's not possible due to the exclusive lease (https://gitlab.com/gitlab-org/gitlab/-/blob/cc723071ad337573e0360a879cbf99bc4fb7adb9/app/uploaders/object_storage.rb#L250).
We will need a way to properly queue both worker jobs in sequence and not in parallel.