Skip to content

Fix logic to determine export status

Aishwarya Subramanian requested to merge concurrent-export-requests into master

What does this MR do?

This MR fixes the way project export states are determined. Currently, project export status returns only none and finished.

  1. When a new project export job is enqueued, an entry is made in the newly created table project_export_jobs => project_id, jid and status is set to started.
  2. The status is changed to finished when the job completes successfully.
  3. The status is changed to failed when the maximum retry for the job fails.
  4. Once every hour, a cron job runs to check for stuck jobs and fails them.
  5. The export job is set to expire after 6 hours.

Changes to export statuses:

  1. It removes the after_export_action from list of possible status as it was a redundant information (more details on thread). The project export would transition from none => started => finished

It also adds regeneration_in_progress state to handle when a project export is initiated after it has been recently exported and is available for download.

Testing scenarios:

Use case Expected behavior Status
Initiate new project export (Happy flow) State moves from none => queued => started => finished
Initiate project export from API when a finished one exists State moves from finished => regeneration_in_progress => finished
Initiate project export from API fails, when a finished one exists State moves from finished =>regeneration_in_progress => finished
Initiate consecutive project exports (more than one user of the project initiates simultaneously) A queued => queued, A starts => started, B queued => queued, B starts => started, A finishes => regenration_in_progress, B finishes => finished
Project export fails (number of exceptions maxes out) State moves from none => queued =>started => none
Fail stuck jobs Sets Status to queued => started => failed
Simultaneous project creations w/ built-in template Creates all projects

Future improvements:

  1. Introduce failed export status

Mentions #32203 (closed)

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team
Edited by 🤖 GitLab Bot 🤖

Merge request reports