Add rate limit handling to imports from GHES
What does this MR do and why?
This MR updates the GitHub import client to wait for rate limits to expire when importing from GitHub Enterprise Server (GHES). Previously, our importer assumed that rate limiting was never enabled on GHES instances, when rate limiting is only disabled by default on GHES instances. So when importing from rate-limited instances, nothing stopped executing import workers even though API requests to fetch resources would raise errors, and eventually the workers would exhaust their retries and fail to import all expected project resources. The consequences were more pronounced with lower rate limits.
References
- Resolves #621582 (closed)
- How to request access and use our GHES test instance: https://gitlab.com/gitlab-org/foundations/import-and-integrate/team/-/blob/main/importers/github_enterprise_server_test_instance.md
- GitHub rate limiting documentation for GHES: https://docs.github.com/en/enterprise-server@3.21/rest/using-the-rest-api/rate-limits-for-the-rest-api?apiVersion=2026-03-10
Screenshots or screen recordings
This MR does not make any changes to the UI, but its effects are visible on the project import history page:
| Before | After |
|---|---|
![]() |
![]() |
![]() |
How to set up and validate locally
- Set up the GHES test instance and request access to a user on the instance according to https://gitlab.com/gitlab-org/foundations/import-and-integrate/team/-/blob/main/importers/github_enterprise_server_test_instance.md
- Using the
ghe_root_site_admincredentials located in the shared engineering 1Password vault, log into the GHES management console, enable rate limiting, and set the limit to something low.- Note: This feature makes it so that importing may take a long time. With
corelimits to 50 requests per hour, thevuejs-coreproject took about 3 hours to import into my GDK. I didn't see a way to change the time window for rate limits, so each time a limit is reached means an additional hour to import.
- Note: This feature makes it so that importing may take a long time. With
- Import one of the projects on the instance using the GitLab API
- Verify that the import completed as expected with all records imported. Verify that rate limits were encountered by checking
importer.logfor entries containing"severity": "INFO"and"exception_class":"Gitlab::GithubImport::RateLimitError" - Verify imports from https://github.com are unaffected
MR acceptance checklist
Evaluate this MR against the MR acceptance checklist. It helps you analyze changes to reduce risks in quality, performance, reliability, security, and maintainability.
Related to #621582 (closed)


