Skip to content

GitHub & Gitea importer filtering

George Koltsov requested to merge georgekoltsov/add-github-importer-filtering into master

What does this MR do?

This MR adds filtering functionality to GitHub & Gitea importers. Please see the gif below.

Note on filtering being done on our side: Octokit client does not provide an easy way of filtering all user's repositories (see https://octokit.github.io/octokit.rb/Octokit/Client/Repositories.html#repositories-instance_method & https://developer.github.com/v3/repos/#list-your-repositories). Instead, GitHub provides Search API and has search_repositories functionality (see https://octokit.github.io/octokit.rb/Octokit/Client/Search.html#search_repositories-instance_method). However, this is not exactly what is needed, since search is performed on the entire GitHub. It is possible to scope down this search to user/organisation, but I encountered 2 problems with it:

  • We need to know if provided access token belongs to user or organisation. I could not find this functionality available in Octokit client (this is probably the biggest showstopper from using search API)
  • When scoping by user I was getting unexpected results. Results did not contain repositories I was a collaborator in, which is different to what we currently return (client.repos returns both your repos and all repos you are collaborator in)

Since currently we retrieve all repositories from GitHub anyway, I decided to go with least traction solution of doing filtering after fetching all repos from GitHub.

This MR originally only targeted GitHub, but frontend and backend code is shared between GitHub and Gitea, so I added a few changes to support both importers.

EE Port: https://gitlab.com/gitlab-org/gitlab-ee/merge_requests/16185

Screenshots

Before

image

After

githubish.mov

Does this MR meet the acceptance criteria?

Conformity

Performance and Testing

Edited by 🤖 GitLab Bot 🤖

Merge request reports