Missing objects for imported Github repo when github api rate limit is reached
Currently in case when GitHub api rate limit is reached during import, imported project will result in missing objects like issues, mrs and comments.
Investigate if hitting rate limit is handled properly
Reproducing locally
To reproduce issue locally on running GDK instance which is available via 127.0.0.1:3000
following command can be used from within qa
folder of gitlab
project:
$ QA_DEBUG=1 QA_LARGE_IMPORT_GH_TOKEN=test_client_token GITHUB_ACCESS_TOKEN=token_passed_to_gitlab bundle exec rspec qa/specs/features/api/1_manage/import_large_github_repo_spec.rb
Randomized with seed 1227
Manage
Project import
...
where both tokens just have to be valid github api tokens. It will work with one single token but then test client will probably run in to api rate limit and currently it has no handling for it.
ping @acunskis in slack if You don't have any tokens available.
The test will spit out all the difference via rspec expectations, but in case of many missing objects it might not be very readable. qa/tmp
folder will contain data.json
that contains a more readable diff of source and import.
Proposal
- Update the Octokit Gem to the latest version (
https://github.com/octokit/octokit.rb/releases/tag/v4.23.0) - Make sure the update didn't break anything
- Re-run the
rspec/rspec-mocks
import locally and check if the rate limit issue still exists.