Add Bulk Imports API to view user initiated imports
What does this MR do?
This MR adds new Group Migration (Bulk Import) API in order to allow changes to Group Migration feature (https://docs.gitlab.com/ee/user/group/import/) UI (#285577 (closed)) to show more details back to the user so they get a better understanding on the import progress.
4 GET endpoints for an authenticated user:
- Bulk Import API to view all user initiated imports
- Entities API to view all imports' entities. Each entity represent a group or a project (only groups are supported now, projects to be added later)
- View individual import's information and entities
- Entities include failures information (in order to show user more granular detail in the UI, why it failed)
Mentions #331568 (closed)
Migrations output
Up
== 20210629153519 AddIndexToBulkImportEntitiesOnBulkImportIdAndStatus: migrating
-- transaction_open?()
-> 0.0000s
-- index_exists?(:bulk_import_entities, [:bulk_import_id, :status], {:name=>"index_bulk_import_entities_on_bulk_import_id_and_status", :algorithm=>:concurrently})
-> 0.0046s
-- execute("SET statement_timeout TO 0")
-> 0.0006s
-- add_index(:bulk_import_entities, [:bulk_import_id, :status], {:name=>"index_bulk_import_entities_on_bulk_import_id_and_status", :algorithm=>:concurrently})
-> 0.0081s
-- execute("RESET ALL")
-> 0.0005s
-- transaction_open?()
-> 0.0000s
-- indexes(:bulk_import_entities)
-> 0.0025s
-- remove_index(:bulk_import_entities, {:algorithm=>:concurrently, :name=>"index_bulk_import_entities_on_bulk_import_id"})
-> 0.0053s
== 20210629153519 AddIndexToBulkImportEntitiesOnBulkImportIdAndStatus: migrated (0.0240s)
Down
== 20210629153519 AddIndexToBulkImportEntitiesOnBulkImportIdAndStatus: reverting
-- transaction_open?()
-> 0.0000s
-- indexes(:bulk_import_entities)
-> 0.0034s
-- execute("SET statement_timeout TO 0")
-> 0.0009s
-- remove_index(:bulk_import_entities, {:algorithm=>:concurrently, :name=>"index_bulk_import_entities_on_bulk_import_id_and_status"})
-> 0.0041s
-- execute("RESET ALL")
-> 0.0012s
-- transaction_open?()
-> 0.0000s
-- index_exists?(:bulk_import_entities, :bulk_import_id, {:name=>"index_bulk_import_entities_on_bulk_import_id", :algorithm=>:concurrently})
-> 0.0018s
-- add_index(:bulk_import_entities, :bulk_import_id, {:name=>"index_bulk_import_entities_on_bulk_import_id", :algorithm=>:concurrently})
-> 0.0047s
== 20210629153519 AddIndexToBulkImportEntitiesOnBulkImportIdAndStatus: reverted (0.0188s)
Screenshots (strongly suggested)
Does this MR meet the acceptance criteria?
Conformity
-
I have included changelog trailers, or none are needed. (Does this MR need a changelog?) -
I have added/updated documentation, or it's not needed. (Is documentation required?) -
I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) -
I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) -
I have self-reviewed this MR per code review guidelines. -
This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) -
I have followed the style guides. -
This change is backwards compatible across updates, or this does not apply.
Availability and Testing
-
I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) -
I have tested this MR in all supported browsers, or it's not needed. -
I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed.
Security
Does this MR contain changes to processing or storing of credentials or tokens, authorization and authentication methods or other items described in the security review guidelines? If not, then delete this Security section.
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team
Merge request reports
Activity
assigned to @georgekoltsov
- A deleted user
added databasereview pending label
1 Warning This merge request is quite big (702 lines changed), please consider splitting it into multiple merge requests. 1 Message This merge request adds or changes documentation files. A review from the Technical Writing team before you merge is recommended. Reviews can happen after you merge. Mentioning @gitlab-data/engineers to notify the team about changes to the db/structure.sql file.
Documentation review
The following files require a review from a technical writer:
doc/api/bulk_imports.md
The review does not need to block merging this merge request. See the:
-
Metadata for the
*.md
files that you've changed. The first few lines of each*.md
file identify the stage and group most closely associated with your docs change. - The Technical Writer assigned for that stage and group.
- Documentation workflows for information on when to assign a merge request for review.
Reviewer roulette
Changes that require review have been detected!
Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:
Category Reviewer Maintainer backend Marc Shaw ( @marc_shaw
) (UTC+2, 1 hour ahead of@georgekoltsov
)Bob Van Landuyt ( @reprazent
) (UTC+2, 1 hour ahead of@georgekoltsov
)database Alex Buijs ( @alexbuijs
) (UTC+2, 1 hour ahead of@georgekoltsov
)Mayra Cabrera ( @mayra-cabrera
) (UTC-5, 6 hours behind@georgekoltsov
)~migration No reviewer available No maintainer available To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.
To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.
Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.
If needed, you can retry the
danger-review
job that generated this comment.Generated by
DangerSetting label(s) devopsmanage sectiondev based on ~"group::import".
added devopsmanage sectiondev labels
added 1 commit
- bfed2c9a - Add Bulk Imports API to view user initiated imports
added 1 commit
- 7519d7ab - Add Bulk Imports API to view user initiated imports
added 672 commits
-
7519d7ab...450bd790 - 671 commits from branch
master
- 15750126 - Add Bulk Imports API to view user initiated imports
-
7519d7ab...450bd790 - 671 commits from branch
changed milestone to %14.1
added featureaddition label
added typefeature label
added 1 commit
- be046ad1 - Add Bulk Imports API to view user initiated imports
added 1 commit
- 897dc49c - Add Bulk Imports API to view user initiated imports
added 1 commit
- 43ef87a0 - Add Bulk Imports API to view user initiated imports
added 1 commit
- 3112b1cd - Add Bulk Imports API to view user initiated imports
added 1 commit
- d2713d3b - Add Bulk Imports API to view user initiated imports
added 1 commit
- e4b72fc1 - Add Bulk Imports API to view user initiated imports
- Resolved by Imre Farkas
@kassio may I ask you for the review of this change? I still need to fix CI and add documentation, but besides that it's good to be reviewed. Thanks
requested review from @kassio
- Resolved by George Koltsov
- Resolved by George Koltsov
- Resolved by George Koltsov
- Resolved by George Koltsov
removed review request for @kassio
added 182 commits
-
e4b72fc1...0790bb72 - 181 commits from branch
master
- c107155e - Add Bulk Imports API to view user initiated imports
-
e4b72fc1...0790bb72 - 181 commits from branch
- A deleted user
added documentation label
marked the checklist item I have included changelog trailers, or none are needed. (Does this MR need a changelog?) as completed
marked the checklist item I have added/updated documentation, or it's not needed. (Is documentation required?) as completed
marked the checklist item I have properly separated EE content from FOSS, or this MR is FOSS only. (Where should EE code go?) as completed
marked the checklist item I have added information for database reviewers in the MR description, or it's not needed. (Does this MR have database related changes?) as completed
marked the checklist item I have self-reviewed this MR per code review guidelines. as completed
marked the checklist item This MR does not harm performance, or I have asked a reviewer to help assess the performance impact. (Merge request performance guidelines) as completed
marked the checklist item I have followed the style guides. as completed
marked the checklist item This change is backwards compatible across updates, or this does not apply. as completed
marked the checklist item I have added/updated tests following the Testing Guide, or it's not needed. (Consider all test levels. See the Test Planning Process.) as completed
marked the checklist item I have tested this MR in all supported browsers, or it's not needed. as completed
marked the checklist item I have informed the Infrastructure department of a default or new setting change per definition of done, or it's not needed. as completed
requested review from @kassio
- Resolved by Imre Farkas
@ngaskill may I ask you to do documentation review of this change? Thanks
- Resolved by Tiger Watson
@alexbuijs may I ask you to do database review of this change? Thanks
requested review from @ngaskill and @alexbuijs
- Resolved by Nick Gaskill
- Resolved by Nick Gaskill
- Resolved by Nick Gaskill