Skip to content

Fix refresh import state JID in GitHubImport Stages

Not all stage classes refresh the import state JID expiration time, and if a stage takes more than 24 hours to go through GitHub's API and import all stage's records, the migration will be incorrectly marked as failed by the StuckProjectImportJobsWorker worker, which will cause the migration to be aborted.

For example, the ImportIssuesAndDiffNotesWorker stage which imports issues and diff notes doesn't call #refresh_jid_expiration after going through GitHub's API, and #refresh_jid_expiration will only be called by the AdvanceStageWorker after all issues and diff notes are processed

Proposed solution

Add a project.import_state.refresh_jid_expiration call to all GitHub Import stages.

The call should be placed at the beginning of the stage execution. If possible, it should be added to the StageMethods module like the example below and removed from the stages classes.

diff --git a/app/workers/concerns/gitlab/github_import/stage_methods.rb b/app/workers/concerns/gitlab/github_import/stage_methods.rb
index 285a28ba9a69..52e6404601ce 100644
--- a/app/workers/concerns/gitlab/github_import/stage_methods.rb
+++ b/app/workers/concerns/gitlab/github_import/stage_methods.rb
@@ -55,6 +55,8 @@ def perform(project_id)
       # client - An instance of Gitlab::GithubImport::Client.
       # project - An instance of Project.
       def try_import(client, project)
+        project.import_state.refresh_jid_expiration
+
         import(client, project)
       rescue RateLimitError
         self.class.perform_in(client.rate_limit_resets_in, project.id)