Skip to content

Implement npm audit endpoints

🍀 Context

While executing npm audit there are two cases:

  1. GitLab npm Registry is neither set globally nor explicitly for npm audit: npm audit --registry <registry>.
  2. GitLab npm Registry is either set globally or explicitly for npm audit: npm audit --registry <registry>.

With (1.) npm will perform a call to registry.npmjs.org with the list of installed packages including all private packages from GitLab or other sources.

With (2.) npm will perform a call to GitLab npm Registry with the list of installed packages including all private packages from GitLab or other sources. GitLab npm Registry doesn't know how to handle the requests from npm audit command and fail.

This MR addresses the (2.)

According to the docs npm audit will create two consecutive requests:

  1. Bulk advisory POST /-/npm/v1/security/advisories/bulk

  2. Quick audit POST /-/npm/v1/security/audits/quick

As of version 7, npm uses the faster Bulk Advisory endpoint to optimize the speed of calculating audit results and the second request is a fallback and only happens in case the first request is failed.

🔎 What does this MR do and why?

This MR introduces four new endpoints: two on the instance level and two on the project level.

Instance level

  1. POST /api/v4/packages/npm/-/npm/v1/security/advisories/bulk

  2. POST /api/v4/packages/npm/-/npm/v1/security/audits/quick

Project level

  1. POST /api/v4/projects/:project_id/packages/npm/-/npm/v1/security/advisories/bulk

  2. POST /api/v4/projects/:project_id/packages/npm/-/npm/v1/security/audits/quick

Depending on the setup of registry url (instance or project prefix) npm audit will perform a call to GitLab npm Registry with the list of installed packages including all private packages from GitLab or other sources. The payload is a JSON but compressed with Gzip. After that we have two scenarios:

  1. The forwarding of request is allowed by user, then we will propagate a request forward to registry.npmjs.org with all information. registry.npmjs.org will return a report about known vulnerability for found packages.

  2. The forwarding of request isn't allowed by user, then we will return [] since we don't scan packages for known vulnerabilities yes.

📷 Screenshots or screen recordings

Reported vulnerability

Screenshot_2022-11-15_at_11.15.33

🔬 How to set up and validate locally

  1. Prepare a package

    def fixture_file_upload(*args, **kwargs)
      Rack::Test::UploadedFile.new(*args, **kwargs)
    end
    
    FactoryBot.create(:npm_package, project: Project.first)
  2. Grab required information

    Packages::Package.last.name

    Write it down somewhere or remember - it will be required at the later step.

  3. Use root user and create a new project, for instance hello-npm, in a new or existing group, for instance gitlab-org.
    For the sake of simplicity I will reference with hello-npm as a project and gitlab-org as a group later, but it can be whatever you choose.

  4. Create a new npm package on your machine

    mkdir hello-npm
    cd hello-npm
    touch .tool-versions
    echo nodejs 16.15.0 > .tool-versions
    npm init

    Note: while generating a new npm package, answer yes/ok on the all questions.

  5. Edit package's name in package.json file to contain GitLab's group and package's name:

    "name": "@gitlab-org/hello-npm"
  6. Authenticate to the Package Registry:

    # Set URL for your scoped packages.
    npm config set @gitlab-org:registry http://gdk.test:3000/api/v4/packages/npm/
    
    # Replace <your_token> with your token.
    npm config set -- '//gdk.test:3000/api/v4/packages/npm/:_authToken' "<your_token>"

    Note: gdk.test:3000 is a host with running GitLab instance.

  7. Install two packages to hello-npm.

    7.1 Private package created in the step (1.) and name taken in the step (2.)

    npm install @gitlab-org/package-1

    7.2 Public package with known vulnerability

    npm install connect@1.8.1

    Note 1: there is a known security vulnerability which was fixed in 1.8.2

    Note 2: while installing a package npm will perform an audit and report any known vulnerabilities

  8. Run audit using GitLab npm Registry and check the report

    npm audit --registry http://gdk.test:3000/api/v4/packages/npm/

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #227756 (closed)

Edited by Dzmitry (Dima) Meshcharakou

Merge request reports