Document background transfers in the projects and groups API

José M. Requena Plensrequested to merge
gitlab-community/gitlab-org/gitlab:jmrp-transfer-api-background-docs into master

What does this MR do?

Since GitLab 19.4, PUT /projects/:id/transfer and POST /groups/:id/transfer schedule the transfer and respond with the project or group as it was before the move. The user documentation says transfers are asynchronous, but the two API sections still read as if the response followed the move, and they do not say how a client learns that the move finished or failed. This MR documents both on the two sections, as master behaves since !251614 (merged), and completes the action and type lists of GET /todos, which leave out transfer_failed among others.

On each transfer section, it says:

  • A successful request returns the object as it was before the move (200 OK for a project, 201 Created for a group, in the sentence the API style guide places after the attributes table), with history items for the feature flag (19.0) and its removal (19.4), and on the project page one for the 19.5 change to the name and path check.
  • How to confirm the move: read the project or group back until its namespace.id or parent_id is the new one.
  • How a failure is reported: a transfer_failed to-do item for the user who asked, and how to list it with GET /todos.

On the to-do page, the action and type values move into footnotes below the table, and the lists gain the eleven values Todo.action_names accepts and the five types TodosFinder.todo_types accepts that the page left out. I kept the page's convention of not marking the Enterprise Edition values.

Evidence, file and line on master at 5ad13467

The transfer runs in the background, and the response shows the object before the move

  • Project: lib/api/projects.rb:1239 (put ":id/transfer") calls enqueue_async_transfer (lib/api/projects.rb:333-342), which calls Projects::TransferService#schedule_async_transfer and presents project unchanged. app/services/projects/transfer_service.rb:40-70 runs ensure_allowed_transfer, moves the project namespace to transfer_scheduled and enqueues Projects::TransferWorker. spec/requests/api/projects_spec.rb:7055 expects 200 OK.
  • Group: lib/api/groups.rb:712 (post ':id/transfer') calls enqueue_async_transfer (lib/api/groups.rb:275-285), which calls Groups::TransferService#schedule_async_transfer (app/services/groups/transfer_service.rb:36-64) and presents group. spec/requests/api/groups_spec.rb:4074 expects 201 Created, which the page now states.
  • Versions: the REST routes became asynchronous behind groups_and_projects_async_transfer in 19.0 (!232368 (merged), for work item 586549; lib/api/projects.rb at v18.11.12-ee has no enqueue_async_transfer, at v19.0.0-ee it has the flag check), and the flag was removed in 19.4 by !250913 (merged) (merge 4b4c25f4, VERSION 19.4.0-pre). The user pages already carry the 19.4 line (doc/user/project/working_with_projects.md:492, doc/user/group/manage.md:179). They date the flag to 18.11 (:491, :178, work item 594575), when it gated only the UI (app/controllers/projects_controller.rb:147 and app/controllers/groups_controller.rb:233 at v18.11.0-ee). The API pages date the REST change to 19.0, so each page is right about its own entry point.
  • POST /groups/:id/projects/:project_id (lib/api/groups.rb:664-677) still calls Projects::TransferService#execute inline and responds after the move, so its section needs nothing.

Which checks run in the request

  • Project: ensure_allowed_transfer (app/services/projects/transfer_service.rb:148-168) refuses a project already in the namespace (line 152), a caller who cannot change the project's namespace (156; the route's authorize! :change_namespace answers 403 first), a caller who cannot transfer projects into the namespace (160), and, since !251614 (merged), a project with the same name or path there (164, project_with_same_name_or_path_exists? at 171-175). The route turns the error into 400 (lib/api/projects.rb:340).
  • In 19.4 the collision check was in #transfer only (app/services/projects/transfer_service.rb at v19.4.0-ee, line 175), which only the worker runs; !251614 (merged) (merge 87449438, VERSION 19.5.0-pre) moved it.
  • Project checks that still run only in the worker, in #transfer (app/services/projects/transfer_service.rb:178-199): a project with the same name or path pending deletion in the namespace (185, 244-250), container registry tags (191, 201-237), npm packages scoped to the old root namespace (193-196), and the save itself (save_project!, 309-313).
  • Group: ensure_allowed_transfer (app/services/groups/transfer_service.rb:233-243, EE adds SAML or SCIM and the top-level group limit at ee/app/services/ee/groups/transfer_service.rb:11-17) runs in the request, including the path collision (namespace_with_same_path?, 299-301). The worker runs it again and then moves the group, so a group transfer can still fail in the background: Groups::TransferService#execute rescues a Service Desk address conflict (79-80, raised at app/services/service_desk/refresh_project_key_address_slugs_service.rb:24, which app/services/routes/rename_descendants_service.rb:83 calls when the group's path changes), validation errors (85-88) and any TransferError (81-84).

How a failure is reported

  • Both workers call create_transfer_failure_todo when execute returns false or raises after the transfer started (app/workers/projects/transfer_worker.rb:73,78, app/workers/namespaces/groups/transfer_worker.rb:71,77), which calls TodoService#transfer_failed (app/workers/concerns/namespaces/transfer_worker_helper.rb:30-31). That item has action transfer_failed (app/models/todo.rb:33,49), project_id set to the project or group_id to the group (app/services/todo_service.rb:224-229), so GET /todos with action=transfer_failed and the project's numeric ID as project_id finds a project's (the filter is an Integer, lib/api/todos.rb:56, so a URL-encoded path is refused), and its target type is Namespace for a group. A later successful transfer by the same user resolves it (transfer_worker_helper.rb:42-46). The item first appeared in 19.3 (TRANSFER_FAILED is in app/models/todo.rb at v19.3.0-ee and not at v19.2.0-ee).
  • The reason: the worker passes service.error to cancel_transfer! (app/workers/projects/transfer_worker.rb:74, app/workers/namespaces/groups/transfer_worker.rb:72), which stores it as transfer_last_error (app/models/concerns/namespaces/stateful/transition_callbacks.rb:75-80). Todo#transfer_failure_reason reads it (app/models/todo.rb:322-326) and Types::TodoType exposes it as transferFailureReason (app/graphql/types/todo_type.rb:71-74, doc/api/graphql/reference/_index.md:64896), both from !251614 (merged) for #597297 (closed). The REST entity has no such field: lib/api/entities/todo.rb exposes body (line 20), which for this action is the destination path (app/models/todo.rb:365-366).

Sending the request again

  • After ensure_allowed_transfer, schedule_async_transfer calls cancel_stale_transfer_state (app/services/projects/transfer_service.rb:132-146, app/services/groups/transfer_service.rb:216-230). It returns early only while a worker holds the object's Gitlab::ExclusiveLease; otherwise it cancels the scheduled state, and the request schedules the transfer again and enqueues a second worker. With the same destination, that worker fails after the first one moved the object ("Project is already in this namespace.", transfer_service.rb:152, or the group's same_parent?) and creates a transfer_failed item.
  • While a worker holds the lease, the state machine refuses schedule_transfer (app/models/concerns/namespaces/stateful.rb:101-103 transitions only from ancestor_inherited or archived), and the request answers 400 "Unable to initiate transfer. The project may already have a transfer in progress." (app/services/projects/transfer_service.rb:49; the group's at app/services/groups/transfer_service.rb:45). spec/requests/api/projects_spec.rb:7104 and spec/requests/api/groups_spec.rb:4089 hold that answer.

The to-do lists

  • lib/api/todos.rb:54 accepts Todo.action_names and lib/api/todos.rb:59 accepts TodosFinder.todo_types.
  • Todo::ACTION_NAMES (app/models/todo.rb:36-50) has 13 values, and EE adds seven (ee/app/models/ee/todo.rb:8-16). The page listed nine; it left out review_requested, review_submitted, ssh_key_expired, ssh_key_expiring_soon and transfer_failed, and on EE okr_checkin_requested, added_approver, duo_pro_access_granted, duo_enterprise_access_granted, duo_core_access_granted and duo_workflow_input_required.
  • TodosFinder::TODO_TYPES (app/finders/todos_finder.rb:28-31) has 10 types and EE adds five (ee/app/finders/ee/todos_finder.rb:7-8). The page listed ten; it left out WorkItem and Key, and on EE User, ComplianceManagement::Projects::ComplianceViolation and Ai::DuoWorkflows::Workflow.
What I left out of the pages
  • After the technical writing review, the request checks that answer 400 Bad Request, what sending the request again does, and the GraphQL transferFailureReason field. The evidence above still describes them.
  • TodoService keeps one pending item per user, target and action unless the action is in Todo::ACTIONS_MULTIPLE_ALLOWED (app/services/todo_service.rb:399-407, app/models/todo.rb:52), so a second failure while the first item is pending adds nothing, and transferFailureReason is the namespace's last error, which the next scheduled transfer clears (transition_callbacks.rb:27-28). Both are true and I did not think either belongs on the reference page; I can add them if you think they do.
  • A second worker that starts while the first one holds the lease does nothing (app/workers/projects/transfer_worker.rb:113-122), so a resend sometimes produces no second transfer at all.
  • Carrying the reason on the REST to-do entity would be an API change, not a docs one.
Overlap with open merge requests

I checked the diffs of all 555 open merge requests labeled documentation, and of the open merge requests a title search for transfer, to-do and the two API pages returns, for changes to these three pages, and again the 93 open merge requests updated after that check. Eight change doc/api/projects.md or doc/api/groups.md (!251281, !251599 (closed), !254980 (merged), !259313 (merged), !259314 (merged), !259316 (merged), !259710 (merged), !260133 (merged)), all in other sections, and none changes doc/api/todos.md.

Lint
  • markdownlint-cli2 0.23.2 with the repository configuration: 0 issues on the three pages.
  • Vale 3.21.0 with the repository .vale.ini, as scripts/lint-doc.sh runs it (--minAlertLevel error): 0 errors. At every level compared with master: no new alerts, and both transfer pages' reading level goes down slightly (projects 7.89 to 7.81, groups 7.45 to 7.39), and the to-do page's from 6.19 to 5.36.
  • lychee --offline --no-progress --include-fragments on the three pages, in the lint image the docs-lint links job uses: 0 errors, including the new anchors.
Where this comes from

I maintain gitlab-mcp-server, an MCP server that exposes the GitLab API to AI assistants. Its project and group transfer tools read the object back until it moves, and look for a transfer_failed to-do item on each read that finds it unmoved, because the response alone says neither, and it took reading the routes and the workers to learn that. The project keeps a record of everything it finds in its dependencies in upstream-bugs.md, and this MR is the docs change that record proposed, adjusted to what !251614 (merged) changed.

The projects and groups pages belong to Tenant Scale: Organizations and the to-do page to Plan: Work Items. In lib/tasks/gitlab/tw/codeowners.rake the technical writers for those groups are z_painter and brendan777, so they seem the right reviewers.

Related issues

Related to #597297 (closed)

Author's checklist

If you are a GitLab team member and only adding documentation, do not add any of the following labels:

  • ~"frontend"
  • ~"backend"
  • ~"type::bug"
  • ~"database"

These labels cause the MR to be added to code verification QA issues.

Reviewer's checklist

Documentation-related MRs should be reviewed by a Technical Writer for a non-blocking review, based on Documentation Guidelines and the Style Guide.

If you aren't sure which tech writer to ask, use roulette or ask in the #docs Slack channel.

  • If the content requires it, ensure the information is reviewed by a subject matter expert.
  • Technical writer review items:
    • Ensure docs metadata is present and up-to-date.
    • Ensure the appropriate labels are added to this MR.
    • Ensure a release milestone is set.
    • If relevant to this MR, ensure content topic type principles are in use, including:
      • The headings should be something you'd do a Google search for. Instead of Default behavior, say something like Default behavior when you close an issue.
      • The headings (other than the page title) should be active. Instead of Configuring GDK, say something like Configure GDK.
      • Any task steps should be written as a numbered list.
      • If the content still needs to be edited for topic types, you can create a follow-up issue with the docs-technical-debt label.
  • Review by assigned maintainer, who can always request/require the reviews above. Maintainer's review can occur before or after a technical writer review.
Edited by José M. Requena Plens

Merge request reports