Project import/export: allow interrupted project import to be restarted

Problem

Project import with file-based import/export gets interrupted/restarted because of deployments or infra config changes that require restarts or possibly other reasons.

Proposed solution

• We should resume the import process from the point it was interrupted. To do that, we can store on Redis the import process progress and use that information to resume the process when the job restarts.

• On Direct Transfer, we did something similar. For example, for the NDJSON files, we store the last processed line and continue from that line when jobs are restarted.

• From Rodrigo's comment:

  • For all restore executions, we should wrap a condition to check if the restorer was executed
  • The project_tree, uploads_restorer, lfs_restorer will require more checks.
  • For the project_tree restore, which processes several NDJSON files, we need to store the line number of the last processed line and the file name after processing all lines. When resuming the job, we should skip all processed files, and continue from the last processed line.
  • For the uploads_restorer and lfs_restorer, we should store the last importer file. When resuming the job, skip all previous processed files.