## Summary The purpose of this issue is to move an ongoing Slack discussion into a single, trackable place and involve Product in clarifying: - The **future of project file-based export/import** in the context of **Offline Transfer (OT)** and **Direct Transfer (DT)**. - Whether the **instance admin token requirement** for project file imports is still appropriate, given newer mechanisms like **placeholder user mapping / UCM** and **Enterprise Users**. - How this impacts **Congregate ADO → GitLab** migrations that currently depend on file-based export/import. [Original Slack thread](https://gitlab.slack.com/archives/C04RDL3MEH5/p1772535449281139) (expires 2026-06-05). - Participants in the original discussion (non-exhaustive): - **Fabian Koh** – PS perspective and customer/partner use cases. @fabiankoh - **Petar Prokić** – Congregate and ADO migration flows. @pprokic - **Sam Word** – DRI for Offline Transfer, import team perspective. @SamWord - **Rodrigo Tomonari** – feasibility and effort for UCM in file imports, short-term workarounds. @rodrigo.tomonari - **Michael Leopard** – Congregate implementation details and scalability of workarounds. @leopardm - **Thiago Figueiro** – Import group EM, prioritization and alignment with group strategy. @thiagocsf ## Background Key points from the Slack thread: - **Current situation**: - Project file-based export/import APIs require an **instance admin token** to ensure correct mapping of user contributions. - This applies to both **GitLab.com** and **self-managed** instances. - This requirement is a blocker/complication for some **SaaS migrations**, which must currently be run by GitLab internal PS due to security. - **Newer approaches**: - **Direct Transfer (DT)** and other importers (e.g., Bitbucket/GitHub importers) use **placeholder users** and **user consent mapping (UCM)**, allowing: - Group/instance Owners (not just admins) to perform user mappings. - Mapping to real users on SaaS while preserving security through a consent flow. - **Enterprise Users** and verified domains further constrain mappings in a safe way. - **Strategic direction**: - File-based project export/import is **not on the strategic roadmap** and is expected to be **deprecated**. - The Import team is focusing on **Offline Transfer (OT)** (built on DT’s architecture) as the long‑term “file-based” solution. - There is limited capacity to: - Make OT **backward compatible** with the current project export format. - Implement UCM and related flows in the **legacy file-based import/export** path. - **Congregate and ADO**: - **Congregate’s ADO → GitLab migrations** currently: - Produce **custom project exports** in the GitLab project export format, and - Use the **file-based import/export APIs** as the execution mechanism. - Partners see increasing demand for ADO take-outs to GitLab SaaS, but: - The **admin token requirement on GitLab.com** is a major friction point. - If OT deprecates file-based import/export and is **not backward compatible**, Congregate will need to **refactor ADO code** to target OT/DT schemas instead of the legacy export format. ## Problem 1. **Admin token requirement for project file import** - For GitLab.com, it is **not feasible** for customers/partners to obtain an instance admin token. - This limits who can run large-scale migrations, forcing more work through GitLab PS. - At the same time, DT and other importers already allow **non-admin** owners to perform user mappings using placeholder users and UCM, which appears to relax the original security motivation for the admin-only constraint. 2. **Congregate dependency on legacy file-based import/export** - Congregate’s ADO migrations rely on current **project export/import** behavior. - Planned **deprecation** of file-based import/export and potential **schema differences** in OT/DT would require Congregate and similar tooling to pivot to new schemas and APIs. - There’s an open question about how much **backward compatibility tooling** (if any) the Import team should provide versus leaving this entirely to external orchestrators like Congregate. 3. **Short-term vs long-term trade-offs** - Implementing **UCM in project file imports** (and/or relaxing admin requirements) could unblock important customer scenarios in the short term, but: - This appears to be **non-trivial work**, closer in effort to OT-scale changes. - It may **compete directly** with OT roadmap capacity. - Alternatively, asking PS/partners to: - Continue using workarounds (e.g., modifying `project_members.ndjson` at scale via Congregate scripts), or - Wait for OT and then refactor everything to new schemas, could leave a gap for **current ADO → GitLab.com migrations**. ## Options discussed so far From Slack, roughly grouped: 1. **Stay the course: focus on OT/DT, minimal changes to legacy file-based** - Treat file-based export/import as effectively **legacy**, with no substantial new development. - Invest all available capacity into OT (and DT), making them the primary path for: - Self-managed → GitLab.com - Self-managed → self-managed - Expect Congregate (and other tools) to: - **Pivot to OT/DT schemas** in the future, and - Use legacy file-based flows **as-is** in the interim. 2. **OT backward compatibility tool or behavior** - Support **import of existing project export archives** via OT, so tools like Congregate can simply: - Generate existing project exports, and - Ask OT to import them. - Current view from engineering: - Making OT backward compatible in this way is **unlikely in the near term**, due to capacity. - OT was intentionally **decoupled from Congregate** and not designed around an external orchestrator. 3. **Relax admin requirement / introduce UCM in project file imports** - Revisit whether an instance **admin token** is still strictly required for project file imports, given: - DT, BB, GH importers use **placeholder user mapping and UCM** without an admin token. - Enterprise Users + verified domains further mitigate mapping risks. - Possible variants: - Implement **UCM for project file imports**, making user-mapping behavior consistent with DT. - Add special handling for **GitLab.com** (where admin tokens are impossible and `public_email` may not be set) to: - Allow mapping via Enterprise Users API and verified domains. - This is considered a **high-effort change**, closer in scope to OT itself. 4. **Tooling/workaround via Congregate** - Continue to treat legacy file-based export/import as-is, but: - Enhance Congregate (or similar tools) to **mass-edit project archives**, e.g.: - Add missing emails to `project_members.ndjson` before import, when public emails are not available on the source. - This is known to **work technically**, but is: - Operationally messy, especially at scale (hundreds of exports), - Another layer of “glue” that PS/partners must maintain. ## Questions for Product To move forward, we need Product input on the following: 1. **Admin requirement and security model** - Given the current **placeholder user + UCM + Enterprise Users** model in DT and other importers: - Do we still consider the **instance admin token requirement** for project file imports on GitLab.com/self-managed to be necessary for security? - If not, what level of change (e.g., introducing UCM to file imports, relaxing admin only for certain flows, GitLab.com exceptions) would be acceptable on the roadmap? 2. **Role of project file import/export vs OT/DT** - Do we see project file-based import/export as: - Strictly **legacy and to-be-deprecated**, with no new investment, or - A candidate for **targeted improvements** (e.g., UCM, reduced admin requirement) to support interim use cases? - How strongly do we want to steer customers and partners toward **DT/OT-only flows** in the near to mid term? 3. **OT backward compatibility and external orchestrators** - Should the Import team: - Consider any **backward compatibility tooling** (scripts or features) to help with existing export archives, or - Explicitly consider this **out of scope**, leaving it entirely to PS/partners (e.g., Congregate) to adapt? - Is there a **business justification** (e.g., size of ADO → GitLab.com opportunity, PS revenue, strategic migrations) that would warrant: - Either investing in OT backward compatibility, or - Investing in UCM/relaxed admin requirements for legacy file imports? 4. **Short-term guidance for ADO → GitLab.com migrations** - For the next **6–12 months**, what should we recommend to: - Partners and PS teams doing **ADO → GitLab.com** take-outs? - Specifically: - Should they continue to rely on **file-based export/import + workaround tooling** (e.g., editing `project_members.ndjson` at scale)? - Should we encourage **early adoption of DT/OT** as they become available for these flows, and plan Congregate refactors accordingly? - Are there any **“must-have” changes** (e.g., loosening admin requirements) that we should prioritize to unblock these migrations? ## Proposed next steps - **1. Confirm product DRI and label the issue** - Tag the Product Manager responsible for **Import/Offline Transfer** to own the decision and follow-up. - @derekferguson @sranasinghe - **2. Product decision(s)** - Decide on: - Whether to **pursue any change** to admin requirements / UCM in file imports. - Whether to **invest in OT backward compatibility tooling**. - What **short- to mid-term guidance** we officially provide for ADO → GitLab.com migrations using Congregate. - **3. Follow-up implementation issues** - Depending on the outcome, write issues for: - Any Import team engineering work (OT, DT, or file import changes). - Any required updates to Congregate / migration tooling. - **Documentation updates**: - Project export/import docs https://docs.gitlab.com/api/project_import_export/ https://docs.gitlab.com/user/project/settings/import_export/#when-migrating-from-gitlab-self-managed-to-gitlabcom - Import mapping and Enterprise Users docs https://docs.gitlab.com/user/import/mapping/ https://docs.gitlab.com/user/enterprise_user/ - OT/DT/version support docs https://docs.gitlab.com/user/group/import/direct_transfer_migrations/#versions ## References - Slack thread: https://gitlab.slack.com/archives/C04RDL3MEH5/p1772535449281139 - Congregate ADO migration implementation and prior discussion: - https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/issues/1329#note_3127387498 - https://gitlab.com/gitlab-org/professional-services-automation/tools/migration/congregate/-/tree/master/congregate/migration/ado?ref_type=heads - Offline Transfer / Congregate decoupling note: - https://gitlab.com/gitlab-com/content-sites/handbook/-/merge_requests/12056#note_2417309386