GitHub Enterprise imports with LFS files fail if LFS files are stored on a different hostname
Summary
If someone is importing from GitHub Enterprise and they have LFS objects stored in a different server or hostname, GitLab will fail to import the LFS objects due to missing authentication but still mark the import successful.
Steps to reproduce
- Have a GitHub enterprise server with LFS objects stored elsewhere (per the GitHub docs)
- Make a GitHub access token with at least the
repoandread:orgscopes - Use the access token to import the project to GitLab
- Observe the import succeeds but LFS files are not available. Logs indicate a 404 when trying to download the LFS objects
Example Project
- I didn't have a GitHub enterprise server to replicate with, this was reported by a customer
What is the current bug behavior?
When GitLab imports LFS objects, it fails to add authentication headers to the request if the LFS objects are not stored on the same hostname.
What is the expected correct behavior?
Requests to download LFS objects should be authenticated even if they're not going to the same hostname.
Relevant logs and/or screenshots
This is the 404:
{"severity":"ERROR","meta.caller_id":"Gitlab::GithubImport::ImportLfsObjectWorker","meta.feature_category":"importers","meta.root_caller_id":"POST /api/:version/import/github","message":"LFS file with oid [OID] couldn't be downloaded from https://media.[customers-domain]/lfs/[id]/objects/[oid]: Received error code 404"}Output of checks
Results of GitLab environment info
Expand for output related to GitLab environment info
(For installations with omnibus-gitlab package run and paste the output of: `sudo gitlab-rake gitlab:env:info`) (For installations from source run and paste the output of: `sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production`)
Results of GitLab application Check
Expand for output related to the GitLab application check
(For installations with omnibus-gitlab package run and paste the output of:
sudo gitlab-rake gitlab:check SANITIZE=true)(For installations from source run and paste the output of:
sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production SANITIZE=true)(we will only investigate if the tests are passing)
Possible fixes
It looks like in the LfsDownloadLinkService we only add credentials to the request to download the objects if the hostname of the import and the hostname of the LFS object link are the same. In the customer's case they were not (media.github.theirdomain.net vs github.theirdomain.net). So at least in some cases, we're explicitly not adding credentials where we should be.