Implement NPM Remote Cache Entries API
What does this MR do and why?
This MR implements the NPM Remote Cache Entries API for the NPM virtual registry, allowing users to list and delete cached responses from remote upstream registries.
The API provides two endpoints:
-
GET
/virtual_registries/packages/npm/remote/upstreams/:id/cache_entries- Lists all remote cache entries for an upstream with pagination and search support -
DELETE
/virtual_registries/packages/npm/remote/cache_entries/*id- Deletes a remote cache entry (marks aspending_destruction)
Key implementation details:
- Uses the
npm_virtual_registryfeature flag with group as actor - Requires
read_virtual_registrypermission for listing anddestroy_virtual_registryfor deletion - Delete endpoint uses base64-encoded ID (group_id + iid format)
- Follows the Maven implementation pattern from
ee/lib/api/virtual_registries/packages/maven/cache/entries.rb
References
Closes #585862 (closed)
Part of #549804 (NPM virtual registry: manage APIs)
Database
The virtual_registries_packages_npm_cache_remote_entries table is empty in production. It was introduced in 18.8 and the npm_virtual_registry feature flag (introduced in this MR) is disabled by default. No code path populates this table yet. The table size is declared as small.
Note: I created several records (1000 rows) to show how the indexes will be used "at scale".
Query plans
This MR introduces 3 new scopes on virtual_registries_packages_npm_cache_remote_entries:
1. Scope search_by_relative_path
search_by_relative_path - fuzzy search using trigram index (gin_trgm_ops).
Query plan Postgres.ai: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/48991/commands/146542
2. Scope for_group
-
for_group- filter bygroup_id(part of the composite primary key)
Query plan Postgres.ai: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/48832/commands/146300
3. Scope order_iid_desc
-
order_iid_desc- order byiidDESC (part of the composite primary key)
Query plan Postgres.ai: https://console.postgres.ai/gitlab/gitlab-production-main/sessions/48832/commands/146301
Additional notes
The table already has appropriate indexes for these scopes:
- A trigram GIN index on
relative_path(i_vreg_pkgs_npm_cache_remote_entries_on_relative_path_trigram) - We can see it work in one specific partition after creating 1000 rows. -
group_idandiidare the composite primary key, sofor_groupandorder_iid_descare covered
How to set up and validate locally
-
Enable the feature flag:
Feature.enable(:npm_virtual_registry, Group.find(<group_id>)) -
Ensure the group has the
packages_virtual_registrylicense and virtual registry settings enabled -
Create an NPM virtual registry with an upstream and cache entries
-
Test the list endpoint:
GET
/virtual_registries/packages/npm/remote/upstreams/:id/cache_entriescurl --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.example.com/api/v4/virtual_registries/packages/npm/remote/upstreams/<upstream_id>/cache_entries"*Expectation: it should return the cache_entries successfully.
-
Test the delete endpoint:
DELETE
/virtual_registries/packages/npm/remote/cache_entries/*idcurl --request DELETE --header "PRIVATE-TOKEN: <token>" \ "https://gitlab.example.com/api/v4/virtual_registries/packages/npm/remote/cache_entries/<base64_encoded_id>"*Expectation: the
VirtualRegistries::Packages::Npm::Cache::Remote::Entryrecord should have changed its status topending_destruction
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.