Enqueue a worker to sync a metadata cache

Context

In #393636 (closed) we added a background worker to generate metadata cache and in #393655 (closed) we started to respond with a metadata cache when the metadata endpoint is hit.

The next step is to define places and enqueue a worker to sync a metadata cache.

What does this MR do and why?

This MR enqueues the Packages::Npm::CreateMetadataCacheWorker when npm metadata should be updated:

• A package or a package file was deleted.
• A new package was created.
• A tag was added or deleted.
• A metadata endpoint was hit, but metadata cache was missing.

Since the feature still in development, all changes are gated behind the feature flag.

Screenshots or screen recordings

No.

How to set up and validate locally

This is the mix of steps that should be either executed in rails console (ruby commands) or in the terminal.

1. Enable the npm_metadata_cache feature flag for chosen project.

   Feature.enable(:npm_metadata_cache, Project.first)

   Note: Here I used Project.first but any projects can be used, we need later project.namespace.path and project.id In my case it's a @gitlab-org and 1

2. Publish a new npm package.

   2.1. Create an empty npm package.

   $ mkdir test-metadata-cache && cd $_
   $ touch .tool-versions
   $ echo nodejs 18.16.0 > .tool-versions
   $ npm init

   2.2. Create .npmrc file in the same directory with a project -level endpoint.

   $ touch .npmrc
   $ echo '@scope:registry=http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/' >> .npmrc
   $ echo '//gdk.test:3000/api/v4/projects/<project_id>/packages/npm/:_authToken="${TOKEN}"' >> .npmrc

   Note: Replace scope with a project's namespace and <project_id> with project's id.

   2.3. Edit package's name to contain project's namespace

   $ sed -i -e 's/test-metadata-cache/@gitlab-org\/test-metadata-cache/g' package.json

   2.4. Publish a package to the Package registry

   $ TOKEN=<your_token> npm publish

   Note: Replace <your_token> with a valid token.

3. Validate if a new entry of Packages::Npm::MetadataCache was created.

   Packages::Npm::MetadataCache.find_by(package_name: '@gitlab-org/test-metadata-cache')

4. Create a request to a metadata endpoint and check if a metadata cache was returned.

   $ curl --header "PRIVATE-TOKEN: <your-token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/<package_name>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_name> with a package's name.

   The easiest way to check if a response comes from a metadata cache is to check last_downloaded_at attribute of a Packages::Npm::MetadataCache entry. We can do that as following:

   Packages::Npm::MetadataCache.find_by(package_name: '@gitlab-org/test-metadata-cache').last_downloaded_at

   The value should match current time.

Now let's check if a metadata cache gets synched after the changes.

1. Add a new tag for the package

   $ TOKEN=<your_token> npm dist-tag add @scope/test-metadata-cache@1.0.0 my-tag

   Note: Replace <your_token> with a valid token and @scope with a project's namespace.

2. Verify if a metadata contains a new tag

   $ curl --header "PRIVATE-TOKEN: <your-token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/<package_name>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_name> with a package's name.

3. Remove the most recently added tag

   $ TOKEN=<your_token> npm dist-tag rm @scope/hello_world@1.0.0 my-tag

   Note: Replace <your_token> with a valid token and @scope with a project's namespace.

4. Verify if a metadata does not contain this tag anymore

   $ curl --header "PRIVATE-TOKEN: <your-token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/<package_name>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_name> with a package's name.

5. Publish two new versions of the package

   sed -i -e 's/1.0.0/2.0.0/g' package.json
   $ TOKEN=<your_token> npm publish
   sed -i -e 's/2.0.0/3.0.0/g' package.json
   $ TOKEN=<your_token> npm publish

   Note: Replace <your_token> with a valid token.

6. Verify if a metadata does not contain this tag anymore

   $ curl --header "PRIVATE-TOKEN: <your-token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/<package_name>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_name> with a package's name.

7. Remove a package file from the most recent package's version

   Packages::Package.find_by(name: "@gitlab-org/test-metadata-cache", version: "3.0.0").id # note down package's id
   Packages::Package.find_by(name: "@gitlab-org/test-metadata-cache", version: "3.0.0").package_files.last.id # note down package file's id

   $ curl --request DELETE --header "PRIVATE-TOKEN: <your_token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/<package_id>/package_files/<package_file_id>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_id> with a package's id and <package_file_id> with a package file's id.

8. Verify if a metadata is up to date - doesn't contain a package with a version 3.0.0.

   $ curl --header "PRIVATE-TOKEN: <your-token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/<package_name>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_name> with a package's name.

9. Remove a package with the version 2.0.0

   Packages::Package.find_by(name: "@gitlab-org/test-metadata-cache", version: "2.0.0").id # note down package's id

   curl --request DELETE --header "PRIVATE-TOKEN: <your_token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/<package_id>"

   Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_id> with a package's id.

10. Verify if a metadata is up to date - doesn't contain a package with a version 2.0.0.

    $ curl --header "PRIVATE-TOKEN: <your-token>" "http://gdk.test:3000/api/v4/projects/<project_id>/packages/npm/<package_name>"

    Note: Replace <your_token> with a valid token, <project_id> with a project's id and <package_name> with a package's name.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

• I have evaluated the MR acceptance checklist for this MR.

Related to #393652 (closed)