Projects that are being imported are marked as failed when using group level custom templates
Summary
As described in this original issue, creating a new project from a group-level project template on gitlab.com sometimes fails with
The repository could not be imported.
Import timed out. Import took longer than 54000 seconds
It is happening due to a race condition described here: #284498 (comment 450304300)
StuckProjectImportJobsWorker
is marking the project as 'failed' without giving an opportunity for import job to run.
The exact flow is the following:
- New project using custom template is requested
- Blank project is then created with
project.import_state = scheduled
-- in order to show 'Import in Progress' screen on the frontend - Since this is a project from template, a new export job is queued for the template http://gitlab.com/gitlab-org/gitlab/blob/master/ee/app/services/ee/projects/gitlab_projects_import_service.rb#L13-13
- Export of a template is executed in 10 minutes
- While that was happening StuckImportJob ran, found project with import state scheduled but without a jid and marked it as failed, regardless of what timeouts we have
- After export completed, an import was queued http://gitlab.com/gitlab-org/gitlab/blob/master/ee/lib/ee/gitlab/import_export/after_export_strategies/custom_template_export_import_strategy.rb#L30-30
- Lastly, the actual import failed to start with import state was marked as failed http://gitlab.com/gitlab-org/gitlab/blob/master/app/workers/repository_import_worker.rb#L46-46
What is the current bug behavior?
Project creation using custom group level project templates sometimes fail, depending on when StuckProjectImportJobsWorker
is ran and when you create a new project.
What is the expected correct behavior?
Project creation should be successful, regardless of when StuckProjectImportJobsWorker
is ran.
Possible fixes
StuckProjectImportJobsWorker
should hold off marking the project without jid as failed and instead wait for some time before doing that, as custom template projects do not have jid immediately, since template project needs to be exported first, which takes time.