Add Cargo crate download endpoint

Tim Rizzirequested to merge
feature/cargo-download-endpoint into master

Summary

Adds the Cargo (Rust) crate download endpoint as part of the Cargo Package Manager MVC.

  • New endpoint: GET /api/v4/projects/:id/packages/cargo/:name/:version/download
  • New finder: Packages::Cargo::PackageFinder — locates a Cargo package by normalized name + version, scoped to a project and installable statuses
  • Registers i_package_cargo_user and i_package_cargo_deploy_token HLL counters and metric YAMLs so track_package_event('pull_package', :cargo, ...) does not raise

All changes remain behind the existing package_registry_cargo_support WIP feature flag (disabled by default), so no changelog entry is included.

Plan context

This is MR 1 of 5 planned for the remaining work in the Cargo MVC. The download endpoint is mechanically equivalent to the RubyGems gems/:file_name download. The full MR breakdown for the remaining work (sparse index, upload authorize, upload publish, GA hardening) is here: #33060 (comment 3360481087)

How the URL is wired

The Cargo CLI fetches config.json (already implemented in !181281 (merged)), which currently returns the same project URL for both dl and api. Per the Cargo spec, the default dl template is {crate}/{version}/download — so the CLI will hit this endpoint as /api/v4/projects/:id/packages/cargo/:name/:version/download.

Test plan

  • bundle exec rspec spec/finders/packages/cargo/package_finder_spec.rb spec/requests/api/cargo_project_packages_spec.rb — 35 examples, 0 failures locally
  • bundle exec rubocop — clean
  • bin/rake gitlab:openapi:v2:generate gitlab:openapi:v3:generate — committed
  • Manual smoke test with the Cargo CLI once a .crate file is seeded in a project (deferred to after MR 4 lands)

Query plan

Validated against Postgres.ai (gitlab-production-main, Postgres 17.5) with a seeded cargo row in project 278964.

Query:

SELECT packages_packages.*
FROM packages_packages
INNER JOIN packages_cargo_metadata
  ON packages_cargo_metadata.package_id = packages_packages.id
WHERE packages_packages.package_type = 15
  AND packages_packages.project_id = 278964
  AND packages_packages.status IN (0, 1, 5)
  AND packages_cargo_metadata.project_id = 278964
  AND packages_cargo_metadata.normalized_name = 'cargo-explain-probe'
  AND packages_cargo_metadata.normalized_version = '1.0.0'
ORDER BY packages_packages.id DESC
LIMIT 1;

EXPLAIN (ANALYZE, BUFFERS):

Limit  (cost=6.65..6.65 rows=1 width=124) (actual time=0.124..0.126 rows=1 loops=1)
  Buffers: shared hit=15
  ->  Sort  (cost=6.65..6.65 rows=1 width=124) (actual time=0.123..0.124 rows=1 loops=1)
        Sort Key: packages_packages.id DESC
        Sort Method: quicksort  Memory: 25kB
        Buffers: shared hit=15
        ->  Nested Loop  (cost=0.58..6.64 rows=1 width=124) (actual time=0.081..0.082 rows=1 loops=1)
              Buffers: shared hit=12
              ->  Index Scan using index_packages_packages_on_project_id_and_package_type on public.packages_packages  (cost=0.44..3.46 rows=1 width=124) (actual time=0.057..0.058 rows=1 loops=1)
                    Index Cond: ((packages_packages.project_id = 278964) AND (packages_packages.package_type = 15))
                    Filter: (packages_packages.status = ANY ('{0,1,5}'::integer[]))
                    Rows Removed by Filter: 0
                    Buffers: shared hit=7
              ->  Index Scan using index_cargo_metadata_on_project_normalized_name_version on public.packages_cargo_metadata  (cost=0.14..3.16 rows=1 width=8) (actual time=0.021..0.021 rows=1 loops=1)
                    Index Cond: ((packages_cargo_metadata.project_id = 278964) AND (packages_cargo_metadata.normalized_name = 'cargo-explain-probe'::text) AND (packages_cargo_metadata.normalized_version = '1.0.0'::text))
                    Buffers: shared hit=5
Settings: work_mem = '230MB', seq_page_cost = '4', effective_cache_size = '472585MB', jit = 'off', random_page_cost = '1.5'

Both sides hit indexes; the metadata Index Cond uses all three columns of index_cargo_metadata_on_project_normalized_name_version. ~0.13 ms total, 15 shared buffer hits, no disk reads.

References

Edited by Tim Rizzi

Merge request reports