RUN AS-IF-FOSS Deploy keys for protected branches - BE

What does this MR do?

This is MR is part of selecting Deploy Keys when creating Protected Branches. This related issue is here.

This backend MR modifies the part of the code that checks whether a user can push to a repository. This is now made a bit more permissive as deploy keys can be selected when defining a protected branch. As written in the issue proposal:

Deploy keys are able to push to protected branch if the owner does not have permission to do, but does have access to the project.

This is the 4th MR related to this issue (out of a total of 6). The MR summary can be seen here.

This feature hides behind the deploy_keys_on_protected_branches feature flag.

Relates to #30769 (closed)

Manual testing and screenshots

Let's take the following scenario: here's Project A's .gitlab-ci.yml file:

stages:
  - test

test git commands:
  stage: test
  before_script:
    - apk update && apk add git openssh
    - eval $(ssh-agent -s)
    - echo "$SSH_PRIVATE_KEY" | ssh-add -
    - mkdir -p ~/.ssh
    - chmod 700 ~/.ssh
    - echo "$SSH_KNOWN_HOSTS" >> ~/.ssh/known_hosts
    - chmod 644 ~/.ssh/known_hosts
    - git config --global user.email "user@example.com"
    - git config --global user.name "User name"
  script:
    - cd ~
    - GIT_SSH_COMMAND="ssh -p 2222" git clone etienne@192.168.1.11:gnuwget/wget2.git
    - cd ~/wget2
    - git checkout test-job
    - echo 'hello' >> README.md
    - git commit -am "Added hello"
    - GIT_SSH_COMMAND="ssh -p 2222" git push origin test-job

As shown above, the job stage will pull the wget2 project, update the README.md file, commit and push to the wget2 repo. The private key of a user (called Lucius) as well as the wget project's host are added as env. variables.

Example 1: The user does not have read-access to the wget2 project (not a member and visibility is private)

Example 2: The user has read-access to the wget2 project but their deploy key is not selected as part of the protected branch.

Example 3: The user has read-access to the wget2 project, is not a member of it but their deploy key is added to the project and selected in the protected branch.

Import/Export testing

Let's say that we have a test-job branch, that has 1 role and 1 deploy key for the Allowed to push level.

After running ::Projects::ImportExport::ExportService.new(project, user).execute and downloading the created tar.gz file, here's what we get:

# tree/project/protected_branches.ndjson

{
  "id":94,
    "project_id":3,
    "name":"test-job",
    "created_at":"2020-09-24T09:09:56.025Z",
    "updated_at":"2020-09-24T09:09:56.025Z",
    "code_owner_approval_required":false,
    "push_access_levels":[{"id":265,
      "access_level":0,
      "created_at":"2020-10-05T13:26:41.306Z",
      "updated_at":"2020-10-05T13:26:41.306Z",
      "user_id":null,
      "group_id":null,
      "deploy_key_id":null},
    {"id":275,
      "access_level":40,
      "created_at":"2020-10-16T12:18:33.086Z",
      "updated_at":"2020-10-16T12:18:33.086Z",
      "user_id":null,
      "group_id":null,
      "deploy_key_id":30,
      "deploy_key":{"id":30,
        "user_id":11,
        "created_at":"2020-07-10T16:29:01.710Z",
        "updated_at":"2020-10-05T09:32:57.017Z",
        "key":"ssh-rsa AAAAB3NzaC1yc2EAAAADAQABAAACAQCuJxu+R4Lf9MJOdMVR8cVm0GtSfkJpiBEWDN/ZOVhUMUgElJdMI58vo0aTaY4nqGswOjWkjer+hlkjfBBLhs5ibY17exx6OyZ+ZAmQd3wCvJvOpzphiharEi4ZqmuUleRh9bSXix9Le9BXg8+Es/2j8cjHLXRnZjOptbAOBC6F/h+vU4FYI1hOrx4nR1fXt+wGQYAaK1tvp97ykB00lOz36StHIbYVZfp51C+DhnzGLEupkw/+2/bKOum1jBb7aiZoPO4DFo7Ph+PoPAqsNljOWOqcF+Q5VkTmsdLnCRWMJX69aKQXef3XfngIgZ7sIfpju3H/aAxtvtfLEapoFDuMDTvhDXi51jggAsAnDEuszIxzKPtNKJwGcpLfU+gXtIQyDjznZy8HzQOOhcWiaail7Ybik0O4V7A94d7TefWxpFKhTaVTRTOasyMpTdVNE7MJKSubYnPMPvMGG7wXL2sG1kvHGuNwPptZHSLyRja0KUqaILGxUQxUq0sQXCH4HJvHsn6nEtnQXBuF2fDq1zz6lFBWy1rfHXt6Xy8wAGGzHk9Oopt4mVX6jgUD45ZT66aroYXJEyiHDllfglxxA3+S/akKlRxm4YXuGhrjoXxkrzydTUIISiNqoJ8Hnt2hsC3e4r6BMfMmIJnrdpFDfv0DS0lkFp1WRB6PuXuQihayCw== lucius.huel@hagenesmitchell.info",
        "title":"Lucius's key",
        "fingerprint":"c9:6b:e6:d6:15:14:a3:a6:4a:88:c9:54:4f:e6:12:49",
        "public":false,
        "last_used_at":"2020-10-16T10:12:30.480Z",
        "fingerprint_sha256":"Qz2ZiDG0Km3W7hK9/BgnjtUFcIyELI+SQ3vGMOHQ8hU",
        "expires_at":null}}],
    "merge_access_levels":[{"id":124,
      "access_level":40,
      "created_at":"2020-09-24T09:09:56.028Z",
      "updated_at":"2020-09-24T09:09:56.028Z",
      "user_id":null,
      "group_id":null}],
    "unprotect_access_levels":[]
}

console:

Manual testing with GitLab Shell v13.11.0

Following the release of this fix, some more manual testing is needed:

• FOSS - Protected Branch with only "No one" as access level -> Can't push to repo.
• FOSS - Protected Branch with "No one" and deploy key. Use of deploy key in CI -> Can push to repo.
• EE - Protected Branch with only "No one" as access level -> Can't push to repo.
• EE - Protected Branch with "No one" and deploy key. Use of deploy key in CI -> Can push to repo.
• EE - Protected Branch with that deploy key's owner selected only (deploy key is not selected). User is added to project as member -> Can push to repo.

2020-10-23: More manual testing about excluding PushAccessLevel records with deploy_key_id

Original project - 1 branch defined with one user, one deploy key and one role:

Export JSON snippet about protected branches (for test-job branch):

{
  "id":106,
    "project_id":3,
    "name":"test-job",
    "created_at":"2020-10-20T12:20:38.880Z",
    "updated_at":"2020-10-20T12:20:38.880Z",
    "code_owner_approval_required":false,
    "merge_access_levels":[{"id":143,
      "access_level":40,
      "created_at":"2020-10-20T12:20:38.882Z",
      "updated_at":"2020-10-20T12:20:38.882Z",
      "user_id":null,
      "group_id":null}],
    "push_access_levels":[{"id":301,
      "access_level":40,
      "created_at":"2020-10-23T10:30:01.716Z",
      "updated_at":"2020-10-23T10:30:01.716Z",
      "user_id":3,
      "group_id":null,
      "deploy_key_id":null},
    {"id":289,
      "access_level":0,
      "created_at":"2020-10-20T12:20:38.885Z",
      "updated_at":"2020-10-20T12:20:38.885Z",
      "user_id":null,
      "group_id":null,
      "deploy_key_id":null}],
    "unprotect_access_levels":[]
}

Imported project:

Does this MR meet the acceptance criteria?

Conformity

• Changelog entry
• Documentation (if required)
• Code review guidelines
• Merge request performance guidelines
• Style guides
• Database guides
• Separation of EE specific content

Availability and Testing

• Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process.
• Tested in all supported browsers
• Informed Infrastructure department of a default or new setting change, if applicable per definition of done

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

• Label as security and @ mention @gitlab-com/gl-security/appsec
• The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
• Security reports checked/validated by a reviewer from the AppSec team

Relates to #30769 (closed)