Allow fork pipelines to run in parent project for external repos

Problem to solve

#11934 (closed) solves forked pipelines in parent project only for local repos. Many users use external repos and need a solution as well.

Intended users

  • Enterprise that uses forking workflow with GitHub and wants to use GitLab CI/CD
  • Open Source projects that live on GitHub but want to use GitLab CI/CD

Further details

Use case:

  1. Some users might want to expose some resources only to internal merge requests, for the purpose of for example provisioning review apps. You need to expose some secrets, but you trust your employees (GitLab), but do not trust external contributors to use these resources (Community Contributions).

Likely each resource should have a flag, fork that indicates that this resource can be used by forks, similar to protected.

This actually leads me to think that protected from boolean should become an enum

  1. In our case we absolutely need to run forks in context of the main repository - while I understand security concerns (by default it should not be possible) we would really need to have a way to enable it.

Another option will be to have separate/dedicated runners in the parent to run fork builds (without any secrets and with more serious limitations imposed such as memory/CPU/max time of build).

This would actually be a really nice compromise between security and functionality. I imagine that we could have in the future much more constrained set of tests that runs for all PRs and more sophisticated (with some secrets) builds triggered either manually or by setting label in github PR or on protected branches only (i.e. after we merge to v1-10-test or master branch in our case).

There could be separate groups of runners for those two types of runs.

The benefit of it that we could provide super-easy way to run tests by the community members (using community-managed resources) and powerful testing tool for the committers (with for example system/integration tests/deployment of documentation/ building images for DockerHub and the like) at the same time.

Proposal

The fork needs to have it's own runners that would be executing these builds, and may not have access to appropriate secrets or other configuration needed to run the pipeline.

Fork users can create pipelines in a parent project, if it fulfils all of the following conditions:

  • The option "Allow fork merge request (or pull request) to run pipelines in the parent project" is enabled in the parent project. This option is off by default.
  • MR/PR has already been created in the parent project. If it doesn't exist, pipelines are not created even if user pushed a branch in a fork repo. i.e. branch pipelines are not supported.
  • The parent/fork projects are public. Private repos are not supported for now.

Process flow overview

Internal Merge requests https://gitlab.com/gitlab-org/gitlab-ee/issues/11934

  1. The parent project enables "Allow fork merge request (or pull request) to run pipelines in parent project" option.
  2. An external user creates/updates a merge request from a forked repo.
  3. MergeRequests::Create/UpdateService runs, which triggers MergeRequests::CreatePipelineService
  4. The pipeline is created on parent project. This pipeline runs on refs/head or refs/merge in parent project.
  5. Runners in the parent project execute the jobs.

External Pull requests https://gitlab.com/gitlab-org/gitlab-ce/issues/65139

  1. User created CI/CD only project from the parent project in GitHub
  2. The parent project enables "Allow fork merge request (or pull request) to run pipelines in parent project" option.
  3. An external user creates/updates a pull request from a forked repo.
  4. created/synchronize events are notified via PullRequestWebhook to GitLab. ProcessGithubPullRequestEventService is executed and SyncGitHubRefWorker runs.
  5. ExternalPullRequests::FetchRefService fetches PR specific refs refs/head (or mirroring) from the parent projects.
  6. ExternalPullRequests::CreatePipelineService creates a pipeline on the fetched ref.
  7. Runners in the parent project execute the jobs.

Permissions and Security

Documentation

Testing

What does success look like, and how can we measure that?

What is the type of buyer?

Links / references

Edited by Orit Golowinski