Set ProjectAttributesPipeline to be a file extraction pipeline
What does this MR do and why?
Related to: #358745 (closed)
- Rename the flag
ndjson_pipeline!
tofile_extraction_pipeline!
- Update the BulkImport pipelines to use the
file_extraction_pipeline!
flag - Add the flag
file_extraction_pipeline!
to the pipelineProjectAttributesPipeline
Context
The ndjson_pipeline!
needs to be added to the BulkImport pipelines that import the data from files that are generated in the source instance. Basically, the flag makes the BulkImports::EntityWorker
to keep re-enqueuing itself until the exported file is ready to be downloaded from the source instance by the target instance.
Initially, the flag was named ndjson_pipeline!
because the re-enqueuing mechanist was only used in pipelines that read the information from .ndjson
files. However now we are also using the re-enqueuing mechanist to read from .json
files, so we decided to rename the flag to file_extraction_pipeline!
.
Besides the flag wasn't present in the ProjectAttributesPipeline
pipeline that reads information from an exported file hence the flag needs to be added to the pipeline.
Screenshots or screen recordings
These are strongly recommended to assist reviewers and reduce the time to merge your change.
How to set up and validate locally
-
Feature.enable(:bulk_import)
. -
Feature.enable(:bulk_import_projects)
. - Create a top level group and project in it.
- Go to
/groups/new#import-group-pane
page and enter instance url and access token (needs to beapi
&read_repository
scope). - Select newly created group and click Import.
- Wait for Group import to complete and verify the imported group and project data.
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.