Fix refresh import state JID in GitHubImport Stages
Not all stage classes refresh the import state JID expiration time, and if a stage takes more than 24 hours to go through GitHub's API and import all stage's records, the migration will be incorrectly marked as failed by the StuckProjectImportJobsWorker worker, which will cause the migration to be aborted.
For example, the ImportIssuesAndDiffNotesWorker stage which imports issues and diff notes doesn't call #refresh_jid_expiration
after going through GitHub's API, and #refresh_jid_expiration
will only be called by the AdvanceStageWorker after all issues and diff notes are processed
Proposed solution
Add a project.import_state.refresh_jid_expiration
call to all GitHub Import stages.
The call should be placed at the beginning of the stage execution. If possible, it should be added to the StageMethods module like the example below and removed from the stages classes.
diff --git a/app/workers/concerns/gitlab/github_import/stage_methods.rb b/app/workers/concerns/gitlab/github_import/stage_methods.rb
index 285a28ba9a69..52e6404601ce 100644
--- a/app/workers/concerns/gitlab/github_import/stage_methods.rb
+++ b/app/workers/concerns/gitlab/github_import/stage_methods.rb
@@ -55,6 +55,8 @@ def perform(project_id)
# client - An instance of Gitlab::GithubImport::Client.
# project - An instance of Project.
def try_import(client, project)
+ project.import_state.refresh_jid_expiration
+
import(client, project)
rescue RateLimitError
self.class.perform_in(client.rate_limit_resets_in, project.id)