Skip to content

NPM package registry: Support full form of the metadata endpoint

🔥 Problem

As stated in https://github.com/npm/registry/blob/master/docs/responses/package-metadata.md, the NPM metadata endpoint can return json response of two types:

  1. Full. All available fields are returned.
  2. Abbreviated. Only a handful set of fields are returned. They are actually those need by $ npm install

In #330929 (closed), we decided to return the Abbreviated form all the times to fix the given ~bug at hand and at the same time not introduce a ~performance issue.

🚒 Solution

Read the accept http header sent by clients and select the Full or Abbreviated form accordingly.

🛑 Blocked by

Sending the full metadata json document for each version of a given NPM package can be problematic as some NPM packages have thousands of versions. This could lead to giantic body responses.

This situation is not unique at GitLab. Here is the full metadata of the npm package on npmjs.org: https://registry.npmjs.org/npm. Yes, that's over 15MB 🙀

To counter this situation, we will need #338485 (closed). A background job can help as it has more time to generate both types of metadata and store them in object storage. Then, on the metadata endpoint, it's just a question to select the proper type and send a redirect to the file on object storage.