Extend request_channel tracking to remaining import types

What does this MR do and why?

A previous MR (!256773 (merged)) added request_channel (api / ui / congregate) to the start_project_import and start_group_import events. This lets us see how an import was started via the UI, the API, or via Congregate. That MR covered Direct Transfer, Offline Transfer, and importers built on Import::BaseService (GitHub, Gitea, Bitbucket, Bitbucket Server, Fogbugz).

This MR covers the rest of the import types:

  • File-based project import (upload, remote URL, S3)
  • Manifest import
  • Bitbucket Cloud import from the UI
  • Git repository by URL, for both new projects and retrying a failed import
  • File-based group import

request_channel is still only sent on start events, and channel detection is unchanged. Web controllers pass :ui directly, and API endpoints call Gitlab::Import::RequestChannel.detect(request), which returns :congregate if the User-Agent starts with GitLabApiClient, or :api otherwise. There is a follow up issue to replace User-Agent sniffing for congregate detection: Replace User-Agent sniffing for Congregate dete... (#630340)

Implementation

Entry point How the channel is captured
Import::GitlabProjectsController (UI upload) Passes request_channel: :ui to Projects::GitlabProjectsImportService, which stashes it before calling Projects::CreateService.
POST /projects/import, /projects/remote-import, /projects/remote-import-s3 Pass detect(request) to Import::GitlabProjects::CreateProjectService (new request_channel: keyword, defaults to :api), which forwards it to Projects::GitlabProjectsImportService (new keyword, defaults to nil so templates and the rake import task keep working as before).
Import::ManifestController and Import::BitbucketController (UI) Stash :ui in the controller before calling their project creator. Bitbucket via the API was already covered through Import::BitbucketService.
ProjectsController#create with an import URL Stashes :ui.
POST /projects and POST /projects/user/:user_id with import_url Stash detect(request) through a small helper.
Retry endpoints: Projects::ImportsController#create (:ui) and POST /projects/:id/import/git (detect) Set import_state.request_channel directly before calling .schedule. The import state usually already exists in these cases, so its after_create callback (see below) won't run.
Import::GitlabGroupsController (:ui) and POST /groups/import (detect) Pass request_channel: to Groups::ImportExport::ImportService (new keyword, defaults to :api), which writes it to GroupImportState before enqueuing GroupImportWorker.

"Stashing" the channel means putting it in the request store. ProjectImportState's after_create callback then copies it into Redis, so it's saved before the import job is enqueued. This matters because the job can start running before Projects::CreateService returns.

GroupImportState gets the same treatment as ProjectImportState: new request_channel= / request_channel methods backed by a Redis key (group_import_request_channel_<group_id>). start_group_import now includes request_channel in its payload, using .compact so it's left out when there's no value. Finish and fail events still don't carry it.

The event YAMLs already declare request_channel, so no YAML changes are needed here.

Note for reviewers: this needs an analytics instrumentation review because the start_group_import payload is changing. Also, some of the controller and request specs for this change can't be run in the author's local environment, so CI is the check for those.

Differences from the issue

  • The issue says GroupImportState#request_channel already exists. It doesn't; this MR adds it.
  • The Bitbucket Cloud UI controller is in scope, since it was listed as a follow-up in the previous MR, even though it's not mentioned in this issue.
  • app/controllers/import/url_controller.rb only validates URLs. Git-by-URL imports actually go through ProjectsController#create, POST /projects, and the retry endpoints.
  • The group import UI controller is Import::GitlabGroupsController. groups/import_export_controller.rb, as named in the issue, doesn't exist.
  • The issue suggests setting import_state.request_channel after the record is created. For new projects, that's too late, since the import job can start before creation finishes. Instead, the channel is stashed before creation, following the same pattern as the previous MR.

References

Parent: #617884 Closes #630178 (closed) Previous MR: !256773 (merged) Congregate header follow-up: #630340

Screenshots or screen recordings

Not applicable. This is a backend-only change with no UI.

How to set up and validate locally

Same setup as the previous MR:

  1. Enable Snowplow Micro in GDK: gdk config set snowplow_micro.enabled true, then gdk reconfigure && gdk restart.
  2. Watch events at http://gdk.test:9091/micro/good.
  3. Start imports through each entry point and check the events:
    • UI: GitLab export upload, Manifest, Bitbucket Cloud, Repository by URL, group export upload.
    • API: use curl, e.g. POST /api/v4/projects/import, POST /api/v4/projects with import_url, POST /api/v4/groups/import.
    • Congregate: repeat the API calls with -H "User-Agent: GitLabApiClient".
  4. Confirm start_project_import and start_group_import carry request_channel as ui, api, or congregate depending on how the import was started, and that finish/fail events don't carry it.

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #630178 (closed)

Merge request reports

Loading
Loading