GitHub import rate limit: ensure we are using GitHub App that is owned by a GitHub Enterprise Cloud

GitHub OAuth access tokens that are associated with a GitHub App that is owned by a GitHub Enterprise Cloud have an hourly rate limit of 15k requests, whereas regular OAuth apps have an hourly rate limit of 5k requests. (source)

We already have GitHub OAuth enabled for GitHub imports on GitLab.com.

To try it out, go to https://gitlab.com/import/github/new and click "Authenticate with GitHub"

The OAuth app that is being used is here. Is it owned by a GitHub Enterprise Cloud org? Can't tell from just looking at it.

If we want to switch to a different OAuth app that is owned by the GitHub Enterprise Cloud org so that we can increase the rate limit from 5k to 15k requests per hour, all that we need to do is create that app and then submit a change request issue for prod to swap the app_id / app_secret credentials for the GitHub OAuth config.

If we switch out the config while an import from GitHub is ongoing, the import may break if the access token needs to be refreshed because the access token would no longer be associated with the client_id of the current OAuth app. I would need to look into the token refresh mechanism to be certain about this.

Update: I looked into the issue of whether changing these credentials and, as long as we don't delete the existing OAuth app, it shouldn't. The reason is that we do not refresh OAuth access tokens used for import (they do not have an expiration). We store them in the session via app/controllers/import/github_controller.rb and use them as long as the import is ongoing without refresh. So any actively used access tokens will remain valid even if GitLab is updated to use a new OAuth app for imports.

Swapping out these credentials should be pretty easy; all that we need to do is create that app and then submit a change request issue for prod to swap the app_id / app_secret credentials for the GitHub OAuth config

How to set up OAuth for GitHub imports locally:

  1. Set up an OAuth app in GitHub and set credentials locally in config/gitlab.yml following docs at https://docs.gitlab.com/ee/integration/github.html
  2. In OAuth config in config/gitlab.yml, make sure you have args: { scope: 'repo,read:org' } } set (as opposed to the scopes that our docs tell you to set, those docs assume this is being used for login not import) so that the correct scopes are requested for import. Scopes are set via app/controllers/concerns/import/github_oauth.rb
  3. Start a new GitHub import: https://gdk.test:3443/import/github/new
  4. Click on "Authenticate with GitHub"

QA Notes

After this configuration change, users should still be able to:

  • Sign in using GitHub (if instance is configured to allow GitHub sign in - for example *.gitlab.com instances)
  • On /-/profile/account you should be able to "disconnect GitHub"
  • And on GitHub https://github.com/settings/apps/authorizations you can revoke the authorization
  • Users should be able to import from GitHub using the OAuth method, regardless of whether they have currently connected their GitHub account or not

But, additionally:

Edited by Luke Duncalfe