[meta] Clarify future of project file import/export, admin token requirement, and Congregate ADO migrations

Summary

The purpose of this issue is to move an ongoing Slack discussion into a single, trackable place and involve Product in clarifying:

  • The future of project file-based export/import in the context of Offline Transfer (OT) and Direct Transfer (DT).
  • Whether the instance admin token requirement for project file imports is still appropriate, given newer mechanisms like placeholder user mapping / UCM and Enterprise Users.
  • How this impacts Congregate ADO → GitLab migrations that currently depend on file-based export/import.

Original Slack thread (expires 2026-06-05).

  • Participants in the original discussion (non-exhaustive):
    • Fabian Koh – PS perspective and customer/partner use cases. @fabiankoh
    • Petar Prokić – Congregate and ADO migration flows. @pprokic
    • Sam Word – DRI for Offline Transfer, import team perspective. @SamWord
    • Rodrigo Tomonari – feasibility and effort for UCM in file imports, short-term workarounds. @rodrigo.tomonari
    • Michael Leopard – Congregate implementation details and scalability of workarounds. @leopardm
    • Thiago Figueiro – Import group EM, prioritization and alignment with group strategy. @thiagocsf

Background

Key points from the Slack thread:

  • Current situation:

    • Project file-based export/import APIs require an instance admin token to ensure correct mapping of user contributions.
    • This applies to both GitLab.com and self-managed instances.
    • This requirement is a blocker/complication for some SaaS migrations, which must currently be run by GitLab internal PS due to security.

  • Newer approaches:

    • Direct Transfer (DT) and other importers (e.g., Bitbucket/GitHub importers) use placeholder users and user consent mapping (UCM), allowing:
      • Group/instance Owners (not just admins) to perform user mappings.
      • Mapping to real users on SaaS while preserving security through a consent flow.
    • Enterprise Users and verified domains further constrain mappings in a safe way.

  • Strategic direction:

    • File-based project export/import is not on the strategic roadmap and is expected to be deprecated.
    • The Import team is focusing on Offline Transfer (OT) (built on DT’s architecture) as the long‑term “file-based” solution.
    • There is limited capacity to:
      • Make OT backward compatible with the current project export format.
      • Implement UCM and related flows in the legacy file-based import/export path.

  • Congregate and ADO:

    • Congregate’s ADO → GitLab migrations currently:
      • Produce custom project exports in the GitLab project export format, and
      • Use the file-based import/export APIs as the execution mechanism.
    • Partners see increasing demand for ADO take-outs to GitLab SaaS, but:
      • The admin token requirement on GitLab.com is a major friction point.
    • If OT deprecates file-based import/export and is not backward compatible, Congregate will need to refactor ADO code to target OT/DT schemas instead of the legacy export format.

Problem

  1. Admin token requirement for project file import

    • For GitLab.com, it is not feasible for customers/partners to obtain an instance admin token.
    • This limits who can run large-scale migrations, forcing more work through GitLab PS.
    • At the same time, DT and other importers already allow non-admin owners to perform user mappings using placeholder users and UCM, which appears to relax the original security motivation for the admin-only constraint.

  2. Congregate dependency on legacy file-based import/export

    • Congregate’s ADO migrations rely on current project export/import behavior.
    • Planned deprecation of file-based import/export and potential schema differences in OT/DT would require Congregate and similar tooling to pivot to new schemas and APIs.
    • There’s an open question about how much backward compatibility tooling (if any) the Import team should provide versus leaving this entirely to external orchestrators like Congregate.

  3. Short-term vs long-term trade-offs

    • Implementing UCM in project file imports (and/or relaxing admin requirements) could unblock important customer scenarios in the short term, but:
      • This appears to be non-trivial work, closer in effort to OT-scale changes.
      • It may compete directly with OT roadmap capacity.
    • Alternatively, asking PS/partners to:
      • Continue using workarounds (e.g., modifying project_members.ndjson at scale via Congregate scripts), or
      • Wait for OT and then refactor everything to new schemas, could leave a gap for current ADO → GitLab.com migrations.

Options discussed so far

From Slack, roughly grouped:

  1. Stay the course: focus on OT/DT, minimal changes to legacy file-based

    • Treat file-based export/import as effectively legacy, with no substantial new development.
    • Invest all available capacity into OT (and DT), making them the primary path for:
      • Self-managed → GitLab.com
      • Self-managed → self-managed
    • Expect Congregate (and other tools) to:
      • Pivot to OT/DT schemas in the future, and
      • Use legacy file-based flows as-is in the interim.

  2. OT backward compatibility tool or behavior

    • Support import of existing project export archives via OT, so tools like Congregate can simply:
      • Generate existing project exports, and
      • Ask OT to import them.
    • Current view from engineering:
      • Making OT backward compatible in this way is unlikely in the near term, due to capacity.
      • OT was intentionally decoupled from Congregate and not designed around an external orchestrator.

  3. Relax admin requirement / introduce UCM in project file imports

    • Revisit whether an instance admin token is still strictly required for project file imports, given:
      • DT, BB, GH importers use placeholder user mapping and UCM without an admin token.
      • Enterprise Users + verified domains further mitigate mapping risks.
    • Possible variants:
      • Implement UCM for project file imports, making user-mapping behavior consistent with DT.
      • Add special handling for GitLab.com (where admin tokens are impossible and public_email may not be set) to:
        • Allow mapping via Enterprise Users API and verified domains.
    • This is considered a high-effort change, closer in scope to OT itself.

  4. Tooling/workaround via Congregate

    • Continue to treat legacy file-based export/import as-is, but:
      • Enhance Congregate (or similar tools) to mass-edit project archives, e.g.:
        • Add missing emails to project_members.ndjson before import, when public emails are not available on the source.
    • This is known to work technically, but is:
      • Operationally messy, especially at scale (hundreds of exports),
      • Another layer of “glue” that PS/partners must maintain.

Questions for Product

To move forward, we need Product input on the following:

  1. Admin requirement and security model

    • Given the current placeholder user + UCM + Enterprise Users model in DT and other importers:
      • Do we still consider the instance admin token requirement for project file imports on GitLab.com/self-managed to be necessary for security?
      • If not, what level of change (e.g., introducing UCM to file imports, relaxing admin only for certain flows, GitLab.com exceptions) would be acceptable on the roadmap?

  2. Role of project file import/export vs OT/DT

    • Do we see project file-based import/export as:
      • Strictly legacy and to-be-deprecated, with no new investment, or
      • A candidate for targeted improvements (e.g., UCM, reduced admin requirement) to support interim use cases?
    • How strongly do we want to steer customers and partners toward DT/OT-only flows in the near to mid term?

  3. OT backward compatibility and external orchestrators

    • Should the Import team:
      • Consider any backward compatibility tooling (scripts or features) to help with existing export archives, or
      • Explicitly consider this out of scope, leaving it entirely to PS/partners (e.g., Congregate) to adapt?
    • Is there a business justification (e.g., size of ADO → GitLab.com opportunity, PS revenue, strategic migrations) that would warrant:
      • Either investing in OT backward compatibility, or
      • Investing in UCM/relaxed admin requirements for legacy file imports?

  4. Short-term guidance for ADO → GitLab.com migrations

    • For the next 6–12 months, what should we recommend to:
      • Partners and PS teams doing ADO → GitLab.com take-outs?
    • Specifically:
      • Should they continue to rely on file-based export/import + workaround tooling (e.g., editing project_members.ndjson at scale)?
      • Should we encourage early adoption of DT/OT as they become available for these flows, and plan Congregate refactors accordingly?
      • Are there any “must-have” changes (e.g., loosening admin requirements) that we should prioritize to unblock these migrations?

Proposed next steps

References

Edited by Thiago Figueiró