Change stage execution order in GitHub Import

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Context

As mentioned in our GitHub Import development docs, GitHub Import imports resources in stages and as described in #431603 (closed), some stages can take a long time to complete because they use a single endpoint to retrieve the data.

Currently, the stages are executed in the following order:

1. 🐰 Stage::ImportRepositoryWorker
2. 🐰 Stage::ImportBaseDataWorker
3. 🐰 Stage::ImportPullRequestsWorker
4. 🐰 Stage::ImportCollaboratorsWorker
5. 🐢 Stage::ImportPullRequestsMergedByWorker
6. 🐢 Stage::ImportPullRequestsReviewRequestsWorker
7. 🐢 Stage::ImportPullRequestsReviewsWorker
8. 🐢 Stage::ImportIssuesAndDiffNotesWorker
9. 🐢 Stage::ImportIssueEventsWorker
10. 🐢 Stage::ImportNotesWorker
11. 🐰 Stage::ImportAttachmentsWorker
12. 🐰 Stage::ImportProtectedBranchesWorker
13. 🐰 Stage::ImportLfsObjectsWorker

🐰 = Fast worker. Uses collection endpoint / 🐢 = Slow worker. Uses single endpoint

Idea

It seems that GitHub Import does not follow the correct order of importance when executing the stages and doesn't take into account how long each stage can take to execute. Migrating important items first can be beneficial to users in case a migration fails in the middle. For instance, the user can decide to use the project without the missing resources if they aren't too important to them.

Proposed solution

Change the execution order of the stages.

1. 🐰 Stage::ImportRepositoryWorker
2. 🐰 Stage::ImportBaseDataWorker
3. 🐰 Stage::ImportProtectedBranchesWorker
4. 🐰 Stage::ImportPullRequestsWorker
5. 🐰 Stage::ImportIssues (Split ImportIssuesAndDiffNotesWorker into two stages)
6. 🐰 Stage::ImportCollaboratorsWorker
7. 🐰 Stage::ImportLfsObjectsWorker
8. 🐢 Stage::DiffNotesWorker (Split ImportIssuesAndDiffNotesWorker into two stages)
9. 🐢 Stage::ImportNotesWorker
10. 🐰 Stage::ImportAttachmentsWorker
11. 🐢 Stage::ImportPullRequestsMergedByWorker
12. 🐢 Stage::ImportPullRequestsReviewRequestsWorker
13. 🐢 Stage::ImportPullRequestsReviewsWorker
14. 🐢 Stage::ImportIssueEventsWorker