Allow importers to be easily stopped
Allow importers to be easily stopped
Most of our importers lack a mechanism that enables the administrator to abort a migration in case of an incident.
For example, during the recent gitlab-com/gl-infra/production#17054 (closed) incident, it was suggested that a JOB detected as a problem be terminated. However, this approach is not ideal because importers usually queue thousands of workers, making it impossible to kill all of them.
The best approach in such a situation is to abort the migration responsible for the problem. Unfortunately, not all importers currently provide a simple mechanism that allows the administrator to stop the migration.
Furthermore, I have detected another problem: not everyone is aware that importers can be disabled via the UI, which is a required short-term solution in case a bug is discovered in one of the importers.
Current
| Importer | Current situation | Corrective action |
|---|---|---|
| FogBugz |
 |
|
| Gitea |
|
|
| GitLab Export |
|
|
| Giteat |
|
|
| GitHub |
failed or canceled
|
|
| BitBucket Server |
 |
Update all workers to check the project import state. This way the migration can be interrupted by just changing the state to failed
|
| BitBucketCloud |
|
Update all workers to check the project import state. This way the migration can be interrupted by just changing the state to failed
|
| Direct Transfer |
 |
Update all workers to check the BulkImport state before executing any task, this way the migration can be interrupted by just changing the state to failed
|
Proposed actions
- Implement a mechanism to stop the migration in all importers
- Document in the runbook how to do that. Also, include in the runbook that importers can be disabled via UI
- Issuegitlab-org/gitlab#438285BacklogCategory:Importers Importer:Bitbucket Cloud backend devops foundations group import and integrate section core platform type feature workflow refinement
- Issuegitlab-org/gitlab#438283Backlog3Category:Importers Importer:Bitbucket Server backend devops foundations group import and integrate section core platform type feature workflow ready for development
- Issuegitlab-org/gitlab#46618517.32Category:Importers Deliverable Importer:Direct Transfer backend devops foundations group import and integrate milestone p1 section core platform type feature workflow complete
- Issuegitlab-org/gitlab#43828117.13Category:Importers Deliverable Importer:Direct Transfer backend devops foundations group import and integrate milestone p1 release post item in review section core platform type feature workflow complete
Activity
@rodrigo.tomonari thank you for this issue. Can this be weighted as one issue or does it require splitting up for BB server and BB cloud and DT?
@m_frankiewicz, I think creating one issue per importer would be better. Should we make this issue an epic?
