Skip to content

GitLab Next

  • Projects
  • Groups
  • Snippets
  • Help
    • Loading...
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in / Register
GitLab
GitLab
  • Project overview
    • Project overview
    • Details
    • Activity
    • Releases
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
    • Locked Files
  • Issues 34,932
    • Issues 34,932
    • List
    • Boards
    • Labels
    • Service Desk
    • Milestones
    • Iterations
  • Merge Requests 1,253
    • Merge Requests 1,253
  • Requirements
    • Requirements
    • List
  • CI / CD
    • CI / CD
    • Pipelines
    • Jobs
    • Schedules
    • Test Cases
  • Security & Compliance
    • Security & Compliance
    • Dependency List
    • License Compliance
  • Operations
    • Operations
    • Metrics
    • Incidents
    • Environments
  • Packages & Registries
    • Packages & Registries
    • Container Registry
  • Analytics
    • Analytics
    • CI / CD
    • Code Review
    • Insights
    • Issue
    • Repository
    • Value Stream
  • Snippets
    • Snippets
  • Members
    • Members
  • Collapse sidebar
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
  • GitLab.org
  • GitLabGitLab
  • Issues
  • #244880

Closed
Open
Opened Sep 03, 2020 by Ben Prescott @bprescott_↙ ☺@bprescott_🎩Developer

npm package API rate limits

Summary

There is an issue when pulling npm packages using GitLab CI. Users are reporting that they are hitting a rate limit and having the request rejected. Self-Managed instances can increaes this limit in their HAProxy settings by adjusting rate_limit_http_rate_per_minute and rate_limit_sessions_per_second. But for GitLab.com, the HAProxy responds with an HTTP status code 429 to API requests that exceed 10 requests per second per IP address.

User feedback

  • The issue was originally reported when someone tweeted about an error interacting with the npm registry
  • "Our project uses npm-scope and it should only fetch a few npm packages from the private repository. But we get the error "E429 - Too Many Requests - GET https://gitlab.com/api/v4/packages/npm/is-absolute-url" for packages which should be fetched from the public repository."

Error

npm ERR! 429 Too Many Requests - GET https://gitlab.com/api/v4/packages/npm/chardet

Steps to reproduce

Example Project

What is the current bug behavior?

Rate limiting occurs

What is the expected correct behavior?

If rate limiting is expected behaviour, then it's documented somewhere like

  • https://docs.gitlab.com/ee/security/rate_limits.html
  • https://docs.gitlab.com/ee/administration/instance_limits.html#package-registry-limits

And some guidance on why it's occurring might also be needed.

Relevant logs and/or screenshots

From job log

Getting source from Git repository
$ eval "$CI_PRE_CLONE_SCRIPT"
Fetching changes...
Initialized empty Git repository in /builds/masksfordocs/toolbox/.git/
Created fresh repository.
Checking out 4ecd9e88 as pedro-gutierrez-okteto...
Skipping Git submodules setup
Restoring cache
Checking cache for ui-17641289-2...
Downloading cache.zip from https://storage.googleapis.com/gitlab-com-runners-cache/project/17641289/ui-17641289-2 
Successfully extracted cache
Downloading artifacts
Downloading artifacts for build_ui (716156553)...
Downloading artifacts from coordinator... ok        id=716156553 responseStatus=200 OK token=
Executing "step_script" stage of the job script
$ cd ${REACT_DIR}
$ npm config set registry https://gitlab.com/api/v4/packages/npm/
$ npm ci
npm WARN prepare removing existing node_modules/ before installation
npm ERR! code E429
npm ERR! 429 Too Many Requests - GET https://gitlab.com/api/v4/packages/npm/chardet
npm ERR! A complete log of this run can be found in:
npm ERR!     /root/.npm/_logs/2020-09-02T14_43_57_064Z-debug.log
ERROR: Job failed: exit code 1

Output of checks

This bug happens on GitLab.com

Job failed 24 hours prior to raising this issue. At the time of raising the issue, GitLab.com was: 13.4.0-pre 5c96eb9a

Possible fixes

Edited Oct 08, 2020 by Tim Rizzi
Assignee
Assign to
13.7
Milestone
13.7 (Past due)
Assign milestone
Time tracking
None
Due date
None
Reference: gitlab-org/gitlab#244880