Design for integrating GitLab CI with GitHub SCM (No mirroring)

Everyone can contribute. Help move this issue forward while earning points, leveling up and collecting rewards.

• Close this issue

Related to https://gitlab.com/gitlab-com/gitlab-OKRs/-/work_items/9875+

Release notes

Problem to solve

Users need a better way of integrating an external SCM tool (GitHub) with GitLab CI than the current mirroring solution.

Why?

Currently, GitLab uses a mirroring process to integrate with GitHub repositories, which has led to performance issues and delays in CI pipeline execution. Additionally, we are seeing a trend that users need enhanced security in this area and do not want to allow GitLab access to their code. ➡️ View the research that backs this up

Intended users

GitHub users who are using it for SCM, but want to integrate with GitLab to use GitLab CI. These personas include:

• Software developers
• Platform engineers
• System administrators

User experience goal

Users should be able to run the CI definition (hosted on GitHub) for a repository hosted on GitHub without compromising security.

Proposal

Design specs:

• Be sure to go through all 3 user flows in design management and read the on-screen notes.
• Figma file:
  • Workflow 1 (Admin): Install the GitHub app to connect with your GitLab instance.
  • Workflow 2 (Platform Eng/Project Maintainer): Connect a GitHub project with GitLab and map users to use GitLab CI/CD.
  • Workflow 3 (Developer): Trigger a pipeline on GitHub and see results on GitLab.

Some architectural design details that impact the UX:

This solution will use the GitHub App to connect GitHub and GitLab. ➡️ View the research that backs this up

We are currently leaning towards Idea C. This offers a new way for users to integrate GitHub SCM and GitLab CI. We need to define the user experience for this approach. Pure CI from External Source (Idea C):

• Implement CI that reads directly from the external GitHub repository.
  1. The gitlab-ci.yml is hosted on Github.com
  2. The runners will be fetching from Github directly

Further details

Permissions and Security

Documentation

Availability & Testing

Available Tier

Feature Usage Metrics

What does success look like, and how can we measure that?

What is the type of buyer?

Is this a cross-stage feature?

What is the competitive advantage or differentiation for this feature?

Links / references

Additional information

Iteration breakdown

First iteration

Should allow users to:

1. Workflow 1: Connect their GitHub repository with a GitLab repository
   1. Install a GitLab app on their GitHub repository/organisation
   2. Verify this connection manually
2. Workflow 2: 1 to 1 user mapping on both platforms to get permission to run a pipeline on GitLab from GitHub
   1. Manually go and create a user on GitLab and link it with GitHub
3. Workflow 3: Trigger a pipeline on GitHub and run it successfully on Gitlab
   1. Get a generic error if the user who triggers the pipeline on GitHub does not exist on the GitLab project connected
4. Workflow 4: Display the status of the pipeline running on GitLab(on GitLab)

Later iteration:

1. Automate verification of connection
2. Automate importing user accounts to GitLab
3. Provide more specific and actionable errors when a connection is broken or user identity fails to map
4. Cross-link the resources on GH and GL. For example, users should be able to navigate to the commits and branches on Gh from GL