Add BulkImports Project Members Migration & Refactor Group Members Migration
- Group Migration -
bulk_importfeature flag. Default enabled (& enabled on .com) - Project Migration -
bulk_import_projectsfeature flag. Default disabled (& disabled on .com)
Mentions #341886 (closed)
What does this MR do and why?
This MR:
- Adds Project Membersip migration, which preserves user memberships from source to destination project (for users with public email info)
- Updates Group Membership migration, to not only import direct memberships, but also handle inherited memberships
- Updates
MembersPipeline&GetMemberQueryGraphQL query, which is used to fetch membership information from source, to be used in both Group & Project Migrations - Updates all
bulk_imports/graphql/*queries to become classes, instead of modules, in order to allow easier implementation ofGetMemberQuerythat caters for both groups and projects (since query differs)
The entire members pipeline flow is split into 3 steps (Extract/Transform/Load) and is the following:
Data extraction ⬇
- For each group/project that needs to be imported, fetch direct & inherited memberships using GraphQL (
GetMembersQuery)
Data transformation 🖌
For each user membership:
- Clean up any prohibited attributes
- Map
public_emailinformation on source to a user on destination, using any confirmed email - If user is found, cache
source_user_id => destination_user_idin redis for future use
Data load 💾
- Create user membership if there is no existing membership
- Create user membership if there is existing membership with lower access level (e.g. a user has Developer inherited membership and has Maintainer direct memberhip).
- Do not create user membership if existing inherited membership has higher access level
- Do not create user membership if user_id is current user id, since current user is already a group/project member
Screenshots or screen recordings
https://monosnap.com/file/QeXr6NeR83o76jTKCqHnyuh7BObujE
How to set up and validate locally
-
Feature.enable(:bulk_import). -
Feature.enable(:bulk_import_projects). - Create a top level group and project in it.
- Invite group and project members with public email information (as well as without)
- Go to
/groups/new#import-group-panepage and enter instance url and access token (needs to beapi&read_repositoryscope). - Select newly created group and click Import.
- Wait for Group import to complete and verify imported group & project members. Users with public emails should be present with the same access levels1
MR acceptance checklist
This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.
-
I have evaluated the MR acceptance checklist for this MR.
Edited by George Koltsov