Add Cargo package publish endpoint

Tim Rizzirequested to merge
cargo-publish-endpoint into master

What does this MR do and why?

This MR completes cargo publish for GitLab's Cargo (Rust) package registry.

It is MR 4 of 5 in the Cargo MVC plan. With it, a user can run cargo publish and cargo install against a project-scoped GitLab Cargo registry, using the standard Cargo CLI. Auth works with a personal access token, a deploy token, or a CI job token.

Everything is behind the package_registry_cargo_support feature flag, which is off by default.

What it adds

  • New endpoint: PUT /api/v4/projects/:id/packages/cargo/api/v1/crates/new.
  • Workhorse route: Workhorse now intercepts the raw publish body and uploads it. This needed a new route in workhorse/internal/upstream/routes.go. Without it, the authorize endpoint from MR 3 never gets called, and Rails never receives the file param.
  • Placeholder package flow: The endpoint can't know the crate name or version until the body is parsed. So it creates a placeholder package named Cargo-Temporary-Package with status processing, attaches the raw body as its file, and queues the existing Packages::Cargo::ExtractionWorker. The already-merged Packages::Cargo::UpdatePackageFromMetadataService then renames the package, writes the metadata, and swaps in the real .crate file.
  • New service: Packages::Cargo::CreateTemporaryPackageService.
  • New constant: Packages::Cargo::TEMPORARY_PACKAGE_NAME. Cargo package names can't have dots, so the shared Temporary.Package default fails validation.
  • New permission: upload_cargo_package, added to the create_package assignable permission.
  • Response shape: The response body is {"warnings":{"invalid_categories":[],"invalid_badges":[],"other":[]}}. The Cargo client fails if it can't parse this.

Two bugs fixed in already-merged code

Both surfaced while testing the new endpoint end to end. This is why the diff touches app/workers/packages/cargo/extraction_worker.rb.

  1. Symbol keys never matched: Sidekiq serializes job arguments to JSON, so hash keys arrive as strings. The worker checked params.key?(:user_id) using a symbol, so it could never resolve the user or deploy token, and returned early. Every published crate would have been stuck in processing status forever, with no error, no status_message, and no retry. The worker now symbolizes the hash.
  2. Symbols aren't valid job arguments: The new service passes string keys instead. A spec asserts this with the param_containing_valid_native_json_types matcher.

References

Requesting review from @10io and @kiran-4444, who worked on the earlier MRs in this stack. In June, @kiran-4444 said they planned to pick up MRs 3 and 4. This MR was picked up after a check-in comment went unanswered.

How to set up and validate locally

Automated tests

  • 153 examples in spec/requests/api/cargo_project_packages_spec.rb pass.
  • 34 examples across the new service spec and the existing worker spec pass.
  • gitlab:permissions:validate, gitlab:openapi:v3:check_docs, gitlab:openapi:v3:validate_tag_docs, and ci:job_tokens:check_policies all pass.
  • Workhorse make vet and go test ./internal/upstream/... pass.
  • The auth matrix is covered: developer, reporter, deploy token, deploy token without write permission, CI job token, anonymous, guest, missing Workhorse headers, and feature flag off.

Manual steps

  1. Rebuild Workhorse. A plain gdk restart won't recompile it. Run make gitlab-workhorse-update, then gdk restart gitlab-workhorse.

  2. In the Rails console, run Feature.enable(:package_registry_cargo_support, Project.find(1)).

  3. Add this to ~/.cargo/config.toml:

    [registries.gdk]
index = "sparse+http://gdk.test:3000/api/v4/projects/1/packages/cargo/"
token = "Bearer <PAT-with-api-scope>"

  4. From a test crate directory, run cargo publish --registry gdk.

  5. From a separate consumer project, run cargo add --registry gdk test-crate, then cargo build.

  6. Confirm the package moved from processing to default status, with the real name and version. Confirm the metadata row exists and the package file is named <name>-<version>.crate.

A package stuck at processing means the worker never ran. A package at error means extraction failed. Check status_message for why.

Known limitation

  • Duplicate and protection checks happen too late on GitLab.com: The endpoint now checks package protection rules up front by reading the crate name from the publish body, but only when the upload is a local file. With object storage direct upload (the GitLab.com setup), Rails never gets a local file to read, so the check falls through to the worker. On GitLab.com, cargo publish still returns 200 for a protected name, and the rejection only shows up later as a package in error status. Duplicate versions are checked only by the worker, everywhere. The Cargo client never sees either message. This differs from npm, which rejects both cases right away. Stored data stays correct: the unique index index_cargo_metadata_on_project_normalized_name_version on packages_cargo_metadata blocks the duplicate from being saved. Moving both checks into the endpoint, for every upload path, is follow-up work in #628146, and it should land before the feature flag rolls out to Beta users.
  • Race condition on simultaneous publishes: Every publish creates a new placeholder package, so the exclusive lease in UpdatePackageFromMetadataService is keyed per placeholder, not per crate version. If two publishes of the same crate and version run at the same time, both can pass the duplicate check. The loser then fails on ActiveRecord::RecordNotUnique, which isn't in Packages::ErrorHandling::CONTROLLED_ERRORS. So the package records Unexpected error instead of Package already exists. This behavior already existed; this endpoint just makes it reachable. It belongs in the same follow-up as the item above.

Router sync

This MR adds PUT /api/:version/projects/:id/packages/cargo/api/v1/crates/new to the HTTP Router in gitlab-org/cells/http-router!1373 (merged) (merged 2026-09-29). The cells-routes:router-in-sync job still fails, so this MR has the pipeline:skip-router-sync label. The router's main holds 7 routes that aren't on GitLab master yet, like /api/:version/orbit/context and /users/sign_up/company. Rebasing won't fix it, since master is missing those routes too. This hits every MR that changes routes and is tracked in gitlab-org/quality/analytics/ci-health-incidents#1336 (closed).

MR acceptance checklist

Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Tim Rizzi

Merge request reports