Serve npm namespace metadata from the project metadata cache
What does this MR do and why?
Serves the npm group and instance metadata endpoints from the project's existing Packages::Npm::MetadataCache when packages in multiple projects are not allowed, behind the npm_namespace_metadata_cache feature flag (gitlab_com_derisk, default off). There is no merge logic: the endpoint reads the cache file that the project endpoint already serves.
When npm_allow_packages_in_multiple_projects is disabled (the default), the group and instance endpoints already return the metadata of a single project:
project_or_nilpicks the project of the most recent package with that name in the namespace.PackageFinder.new(project: project_or_nil)loads only that project's packages.authorize_read_package!(project)checks access to that one project.
That is the same response as GET /projects/:id/packages/npm/:package_name for that project, which is served from the metadata cache. The namespace endpoints rebuild it on every request instead. This MR makes them read the same cache, in the else branch that already handles the single-project case:
- The cache lookup runs after
authorize_read_package!, the same order as the project endpoint inlib/api/npm_project_packages.rb. last_downloaded_atis updated on GET, not on HEAD.- When the cache file does not exist, the endpoint builds the response on the fly and enqueues
Packages::Npm::CreateMetadataCacheWorker(deduplicated withuntil_executing). - When
npm_allow_packages_in_multiple_projectsis enabled, nothing changes. Merging caches across projects is a separate follow-up.
Why the namespace endpoints never used the cache
The cache was designed for the project and instance endpoints in #338485 (closed). The review there decided to start with the project level only, because "at the instance level, the content of the metadata response depends on the set of projects that the user has access to" (comment). !122686 (merged) then limited the cache to endpoint_scope == :project. The instance level was planned as a later iteration and never added.
That concern applies when packages come from multiple projects. At the time, npm_allow_packages_in_multiple_projects was being rolled out, and the rollout was later reverted (#391692). With the flag disabled, the response comes from one project and access is all or nothing, so every user who can read it gets the same document as the project cache.
Why the flag actor is the root namespace
npm_namespace_metadata_cache is checked against group_or_namespace.root_ancestor:
- The instance endpoint already resolves
group_or_namespaceto a top-level namespace from the package scope. Using the root on the group endpoint too gives both endpoints the same actor for the same customer. - One
--group=<root>ChatOps command enables or disables the flag for every subgroup and project under that root. Aprojectactor would need every project listed, and a plaingroup_or_namespaceactor would not apply to requests through subgroup URLs.
Behavior changes to be aware of
- Group and instance cache misses now enqueue
CreateMetadataCacheWorker, as project cache misses already do. Publishing already creates the cache, so most projects have one. - After a publish, the group and instance endpoints serve the previous document until the cache is rebuilt, as the project endpoint does today. A change made while a rebuild is running can be missed until the next rebuild: #631920 (closed).
References
- https://gitlab.com/gitlab-org/gitlab/-/work_items/636660#sign-off-on-entry
- #567311
- Rollout issue: #637327
Screenshots or screen recordings
Not applicable. This MR has no UI changes.
How to set up and validate locally
-
Create a public group and project, and publish an npm package
@<group-path>/helloto the project. Wait for the package to reach thedefaultstatus. Publishing creates the metadata cache. -
In the Rails console, enable the feature flag for the group:
Feature.enable(:npm_namespace_metadata_cache, Group.find_by_full_path('<group-path>')) -
Request the group endpoint and check that the response is sent from the cache file. With local storage, a cache hit returns
Content-Type: application/octet-streamandContent-Disposition: attachment; filename="metadata.json", while the live response isapplication/json. You can also comparedb_countinlog/api_json.logwith the flag on and off:curl -i "http://gdk.test:3000/api/v4/groups/<group-id>/-/packages/npm/@<group-path>%2Fhello" -
Repeat with the instance endpoint:
http://gdk.test:3000/api/v4/packages/npm/@<group-path>%2Fhello. -
Disable the flag and request the endpoints again. The JSON body is the same as the cached response.
-
Clean up:
Feature.remove(:npm_namespace_metadata_cache)
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.