Avoid the copy operation during Generic Package Registry uploads

🔥 Problem

Similar to Avoid copying objects from one bucket to anothe... (#285597 - closed), the package registry can receive uploads for large files. The majority of the package registry uploads (most formats) will use a workhorse direct upload. In this mode, the file is put on object storage in a temporary location and when the upload is confirmed by the backend, the file is moved to its final location (using a copy operation).

The problem is that the GitLab instance can be connected to different object storage providers and that copy operation can take more or less time depending on the file size.

🚒 Solution

  • Start with one package format. We can choose the Generic Package Repository as a starting point.
  • Add a new column named file_final_path to the packages_package_files table with text type and 1024 char length.
  • Pass two new keyword arguments to authorize_workhorse! method.
    • use_final_store_path: true
    • final_store_path_root_id: <project_or_group_id>
  • This change needs a feature flag and close monitoring during the rollout.