ProjectImport: Import remote file from AWS S3

What does this MR do and why?

Add the /projects/remote-import-s3 endpoint to import a project export file direct from AWS S3.

Related to: #348874 (closed)

Implementation Details

I took the opportunity to restructure the code a little bit. Instead of using inheritance to share code among the project creation classes, I refactor it to use composition. This away the code is a bit more extensible and has more independent parts (less coupling)

The original code structure was like

classDiagram
    CreateProjectFromUploadedFileService <|-- CreateProjectFromRemoteFileService

This was replaced by a single entry point Import::GitlabProjects::CreateProjectService, this class is responsible to create a project when the user parameters are valid. Since the user can use different strategies to create/import a project, the requirements for each strategy is implemented in different Import::GitlabProjects::Strategies::* classes, this way the strategy can be changed by dependency injection. A Import::GitlabProjects:: Strategies::* class must implement:

• initialize(current_user:, params:)
• valid? - validate the user parameters
• errors - when invalid, a ActiveModel::Errors object
• project_params - A hash with the specific strategy extra parameters to create the project

Also, because importing from a generic object storage and from AWS S3 have similar validations to be performed, the Import::GitlabProjects::Strategies::RemoteFileValidator was created to validate the remote file content_length and content_type.

At the end, the new code structure looks like:

classDiagram
    class CreateProjectService {
        current_user
        params  
        file_acquisition_strategy
    }
    class FileUpload~FileAcquisitionStrategies~ {
        params
        initialize(current_user:, params:)
        project_params()
        valid?()
        errors()
    }
    class RemoteFile~FileAcquisitionStrategies~ {
        params
        initialize(current_user:, params:)
        project_params()
        valid?()
        errors()
        content_length()
        content_type()
    }
    class RemoteFileS3~FileAcquisitionStrategies~ {
        params
        initialize(current_user:, params:)
        project_params()
        valid?()
        errors()
        content_length()
        content_type()
    }

    CreateProjectService *-- FileUpload
    CreateProjectService *-- RemoteFile
    CreateProjectService *-- RemoteFileS3

    RemoteFile *-- RemoteFileValidator
    RemoteFileS3 *-- RemoteFileValidator

How to test

• Upload a project export on a AWS S3 server, you might use a Cloud Sandbox
• Enabled the feature flag :import_project_from_remote_file_s3
• Then import the file with a request like:

curl --request POST \
  --url "http://localhost:3000/api/v4projects/remote-import-s3" \
  --header "PRIVATE-TOKEN: <your gitlab access key>" \
  --header 'Content-Type: application/json' \
  --data '{
  "name": "Sample Project",
  "path": "sample-project",
  "region": "<Your S3 region name>",
  "bucket_name": "<Your S3 bucket name>",
  "file_key": "<Your S3 file key>",
  "access_key_id": "<Your AWS access key id>",
  "secret_access_key": "<Your AWS secret access key>"
}'

Screenshots or screen recordings

These are strongly recommended to assist reviewers and reduce the time to merge your change.

How to set up and validate locally

Numbered steps to set up and validate the change are strongly suggested.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.