Skip to content
  • Stan Hu's avatar
    Fix project exports clobbering concurrent export paths · 8d5f875c
    Stan Hu authored and Mayra Cabrera's avatar Mayra Cabrera committed
    When a project export completes, it removes everything in
    `Project#import_export_shared.archive_path`, which can erase files
    needed for another ongoing project export. This is problematic for
    custom templates, which exports an existing project to get the most
    recent changes and imports that archive to another project.
    
    To avoid this from happening, we generate a random unique subpath in the
    shared temporary directory so that multiple exports can work at the same
    time.
    
    Previously the path structure was as follows:
    
    1. Project export files stored in:
       /shared/tmp/project_exports/namespace/project/:random
    2. Project export .tar.gz files stored in:
       /shared/tmp/project_exports/namespace/project
    3. Project export lock file:
       /shared/tmp/project_exports/namespace/project/.after_export_action
    
    Now:
    
    1. Project export files stored in:
       /shared/tmp/project_exports/namespace/project/:randomA/:randomB
    2. Project export .tar.gz files stored in:
       /shared/tmp/project_exports/namespace/project/:randomA
    3. Project export lock files stored in:
       /shared/tmp/project_exports/namespace/project/locks
    
    The .tar.gz files are also now cleaned up in the AfterExportStrategy.
    
    Also, ensure import/export path cleanup always happens.
    
    A failure to update the database or object storage shouldn't
    block us from cleaning up stale directories. This is especially
    important to clear out stale lock file and archive paths.
    
    Closes https://gitlab.com/gitlab-org/gitlab-ee/issues/14716
    8d5f875c