Skip to content

Do not save contributors and exports fields in npm metadata

Dzmitry (Dima) Meshcharakou requested to merge 406617-resolve-npm-structure-too-large-exceptions into master

Context

There is a constraint that limits the package_json column of packages_npm_metadata table to 20k bytes.

What does this MR do and why?

Several customers faced the limitation with a large contributors and exports fields, thus let's exclude them from being saved in package_json in packages_npm_metadata table.

Screenshots or screen recordings

No.

How to set up and validate locally

Publish a package with a large contributors field

  1. Create a new npm package @gitlab-org/test

    $ npm init

  2. Set the correct authentication settings in .npmrc

    $ echo @gitlab-org:registry=http://gdk.test:3000/api/v4/projects/1/packages/npm/\\n//gdk.test:3000/api/v4/projects/1/packages/npm/:_authToken="<PAT token>" > .npmrc

    Note: 1 is a project's id that I chose

  3. Download and execute ruby script ruby put_large_contributors.rb attached to the MR in the package's directory. It'll add a large contributors field to the package.json.

  4. Verify that the package can be published.

    $ npm publish

  5. Check the saved package_json field of Packages::Npm::Metadatum using rails console. It doesn't contain contributors information anymore.

    Packages::Package.where(name: "@gitlab-org/test").last.npm_metadatum.package_json

Publish a package with a large exports field

  1. Bump the version of package to 2.0.0 in package.json to avoid any conflicts with existing published packages.

    "version": "2.0.0"

  2. Download and execute ruby script ruby put_large_exports.rb attached to the MR in the package's directory. It'll add a large exports field to the package.json.

  3. Verify that the package can be published.

    $ npm publish

  4. Check the saved package_json field of Packages::Npm::Metadatum using rails console. It doesn't contain exports information anymore.

    Packages::Package.where(name: "@gitlab-org/test").last.npm_metadatum.package_json

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #406617 (closed)

put_large_contributors.rb put_large_exports.rb

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports