## Context Universal Importer (UnImp) is a plugin-based gateway for loading data into GitLab. GitLab owns and maintains the core framework. Plugins target a published schema, and can come from us, the community, Professional Services, or eventually the factory. The focus is ~"Category:Importers" (third-party-to-GitLab: GitHub, Bitbucket, Gitea etc). Migration (GitLab-to-GitLab) is not the focus, but is not ruled out. If we decide a migration feature like Offline Transfer should be the first thing to run on UnImp, that is a call we can make later. ## Problem There is no official, supported way to get data into GitLab. Over time we have accreted several: per-provider importers, rake tasks, NDJSON files, and the REST and GraphQL APIs. Each is a separate path with its own coverage, its own rules, and its own maintenance. The importers share some infrastructure but there is no shared contract for what importable data looks like. The extraction and mapping is written and maintained provider by provider, and nothing outside the monolith can produce data we accept in a defined, validated way. When a provider changes their API or ships a feature, that lands on our roadmap. Coverage is never complete, and the cost grows with every provider we add. The work is repetitive and schema-shaped: read a provider's data, map it to GitLab's model, write it in. A published schema turns that into a contract anyone can target, including the factory. A risk to keep in mind: we are proposing one more way to load data, and we already have several. See [xkcd 927](https://xkcd.com/927/). UnImp only earns its place if it replaces the others rather than joining them. ## Proposal Build the gateway around a published schema, in sequence: 1. **Universal schema.** Define the language-neutral schema for importable data. It does not depend on the Load service or the messaging layer, so it starts first. 2. **Load service.** A service that takes data in the published schema and writes it into GitLab. It is agnostic to who produced the data: the contract is the schema, nothing else. DT and OT already share one pipeline framework (`BulkImports::Pipeline`) over the same loaders. The Load service generalises that idea past the monolith: any producer that emits valid schema is a client. It is greenfield, with loaders reimplemented against the schema rather than ported. Clients of the Load service are extractors: Offline Transfer, Direct Transfer, GitHub, Bitbucket, community plugins, or factory-generated extractors. Which one we wire up first is a roadmap call, not a property of the Load service. 3. **Plugin / extractor framework.** Define how a plugin produces schema-conformant data and how the gateway validates and ingests it. 4. **First extractor.** Prove the framework end to end with one real provider. We'll decide which one during planning. 5. **Self-serve validation.** A tool that lets a contributor check their output against the published schema before submitting, so contributions do not require us to hand-hold each one. This matches the validation done by the Load service. The Load service is in scope for UnImp. The messaging layer it runs on (NATS vs Sidekiq) is a separate decision, tracked in the child epic [&21659](https://gitlab.com/groups/gitlab-org/-/work_items/21659). ## References - Universal Importer concept: [Migrations 5-year strategy](https://docs.google.com/document/d/1-t2y9R-BC0gg0N3wfSPAjhHZYRw-2OkoLZYkyZSBuw0) (private doc).