Upload offline exports directly to configured object storage

What does this MR do and why?

This MR updates the offline transfer flow such that files are uploaded directly to the provided object storage, bypassing Upload / BulkImports::ExportUpload records.

References

#585537 (closed)

How to set up and validate locally

  1. Count the upload records (Upload.count / BulkImports::ExportUpload.count) and note the totals.
  2. Perform an offline export:
#!/bin/bash

curl --request POST \
  --url "http://gdk.test:3000/api/v4/offline_exports" \
  --header "PRIVATE-TOKEN: $GITLAB_DEV_TOKEN" \
  --header "Content-Type: application/json" \
  --data '{
    "bucket": "offline-transfer-export",
    "s3_compatible_configuration": {
      "aws_access_key_id": "redacted",
      "aws_secret_access_key": "redacted",
      "region": "gdk",
      "path_style": true,
      "endpoint": "http://127.0.0.1:9000"
    },
    "entities": [
      {
        "full_path": "flightjs"
      }
    ]
  }'
  1. Count the upload records again to confirm that no upload records have been created.
  2. Confirm that the exported relations are present in your configured object storage.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by James Nutt

Merge request reports

Loading