Skip to content

Track npm structure too large exceptions

Summary

NPM Package uploads were failing with a Validation failed: Package json structure is too large error when a README.md is included in the package. This appears to have started after the packages_npm_abbreviated_metadata was enabled.

Steps to reproduce

While the packages_npm_abbreviated_metadata feature flag is enabled:

  1. Build an NPM package that includes a README.md file
  2. Publish the package to GitLab's NPM Package Repository
  3. Receive this error:
{
    "message": "Validation failed: Package json structure is too large"
}

Solution

When npm metadata creation fails because of the package json metadata being too large, log the error along with the field sizes of the package json metadata. This will help identify the json metadata fields that were responsible for the error.

Once we've collected the data we will open a new issue with a fix for the bug.

Example Project

https://gitlab.com/anton/gitlab-smoke-tests/

What is the current bug behavior?

NPM Packages with large README.md cannot be uploaded.

What is the expected correct behavior?

NPM Packages with large README.md can be uploaded.

Relevant logs and/or screenshots

npm verb stack HttpErrorGeneral: 400 Bad Request - PUT https://gitlab.com/api/v4/projects/27692816/packages/npm/@anton%2fsmoke-test
npm verb stack     at /usr/local/lib/node_modules/npm/node_modules/npm-registry-fetch/check-response.js:95:15
npm verb stack     at processTicksAndRejections (node:internal/process/task_queues:96:5)
npm verb stack     at async publish (/usr/local/lib/node_modules/npm/node_modules/libnpmpublish/publish.js:43:12)
npm verb stack     at async Publish.publish (/usr/local/lib/node_modules/npm/lib/publish.js:128:7)
npm verb statusCode 400
npm verb pkgid @anton/smoke-test@1.0.0-job1785478738
npm verb cwd /builds/anton/gitlab-smoke-tests/npm-registry
npm verb Linux 5.4.109+
npm verb argv "/usr/local/bin/node" "/usr/local/bin/npm" "publish" "--verbose"
npm verb node v16.13.0
npm verb npm  v8.1.0
npm ERR! code E400
npm ERR! 400 Bad Request - PUT https://gitlab.com/api/v4/projects/27692816/packages/npm/@anton%2fsmoke-test

Customer reports

  1. PUT to npm package registry results in 400
  2. Confidential customer ticket: 250925

Output of checks

Results of GitLab environment info

GitLab Enterprise Edition 14.5.0-pre 62abdcc4

Results of GitLab application Check

Expand for output related to the GitLab application check 
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)


(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)


(we will only investigate if the tests are passing)

This appears to be related to these merge requests:

!73639 (merged) !74409 (merged)

Possible fixes

To complete this implementation, please pick up the following MRs and address the feedback in there

Edited by Tim Rizzi