Rate limits for the GitLab package registry
🗞
Current Situation Users are seeing 429 Too Many Request
errors when working with the GitLab package registry.
🔍
Background Users are allowed to use GitLab as a remote registry for their packages. Using the example of NPM, users can set gitlab as their NPM registry:
npm config set registry https://gitlab.com/api/v4/packages/npm/
When the user or CI job runs a command like npm install
, all of the dependencies defined in the given context will be fetched by making calls to the registry. For NPM, this could easily be in the hundreds or thousands. GitLab has logic to forward requests to npmjs.org (the standard public NPM registry) when a package is not found in the GitLab registry. Thus, a user can set a single registry for their project and fetch all dependencies both private and public open source. The NPM client may make a large number of requests in parallel or rapid succession, leading to hitting a rate limit on GitLab.com.
It is theorized that the HAProxy limits defined here, specifically:
default['gitlab-haproxy']['frontend']['api']['rate_limit_http_rate_per_minute'] = '600'
...
default['gitlab-haproxy']['frontend']['https']['rate_limit_sessions_per_second'] = '10'
are the limits causing the error to be thrown.
This has been reported by users for both NPM and NuGet packages.
This behavior has been reproduced in a test project using a CI job.
⭐
Desired Outcome - Validate whether or not an HAProxy rate limit is throwing this error.
- Add rules to allow package GET requests to have higher limits. These URLs would include:
GET requests that begin with:
/api/v4/packages/
/api/v4/projects/<project_id>/packages/
/api/v4/groups/<group_id | group name>/-/packages/
- Alternatively, if such rules are not possible, update the general limit that will allow for the desired behavior.
☑
Acceptance Criteria -
NPM requests no longer hit a 429 error - gitlab-org/gitlab#244880 (closed) is resolved -
NuGet requests no longer hit a 429 error - gitlab-org/gitlab#246538 (closed) is resolved
📓
References - NPM user reported bug
- NuGet user reported bug
- Slack discussion leading to this issue
- Test project with CI job that triggers the error - Feel free to re-run the job to re-trigger the error.