Skip to content
Snippets Groups Projects
Closed Scaling the Security Release process on GitLab.com
  • Scaling the Security Release process on GitLab.com

  • Closed Epic created by Robert Speicher

    Master epic for tracking progress towards:

    • Moving development of security patches from dev.gitlab.org to private projects on gitlab.com.
    • Automate and improve security tasks
    • Include security releases as part of the auto-deploy processes

    References:

    Overview

    This epic is split into multiple epics and issue boards

    On-going

    Epic Issue board Status
    Improvements - Security Release Issue board  :hourglass_flowing_sand:
    Include Security Releases as part of the auto-deploy TBD :hourglass_flowing_sand:

    Completed

    Epic Issue board Status
    Move the security development process as-is to Security --- :white_check_mark:

    Backup 1: First definitions

    Mirroring and syncing

    All protected branches from Canonical are synced to Build via push mirroring.

    :white_check_mark: Existing setup is used.

    Specific protected branches from Canonical (master, X-Y-stable, X-Y-auto-deploy-YYYYMMDD) should be synced to Security via pull mirror, prefixing all branches with security/ (e.g., master on Canonical becomes security/master on Security): gitlab-com/gl-infra/delivery#533 (closed), gitlab-com/gl-infra/delivery#548 (closed)

    All protected branches from Security are synced to Build via push mirroring: gitlab-com/gl-infra/delivery#473 (closed)

    Because both Canonical branches and Security branches (prefixed with security/) are synced to Build, we can perform a regular or security release at any time; one does not block the other.

    Normal release workflow

    :white_check_mark: Nothing changes! :tada:

    Normal releases pull from Build using the X-Y-stable or X-Y-auto-deploy-YYYYMMDD branch, depending on what we're releasing.

    During release, master, stable branches, and tags get pushed to Canonical and Build by release-tools.

    release-tools will:

    1. Pull the Build repository using master
    2. Check and out and pull X-Y-stable[-ee] branch
    3. Compile the changelog on X-Y-stable[-ee] and master
    4. Update the VERSION and component version files
    5. Push X-Y-stable[-ee] and master to the Canonical and Build repositories

    Security release workflow

    :hourglass: This is basically the exact same as the non-security release process, but all branches, even master, are prefixed with security/, and we push to Security and Build (rather than Canonical and Build).

    release-tools will:

    1. Pull the Build repository using security/master
    2. Check and out and pull security/X-Y-stable[-ee] branch
    3. Compile the changelog on security/X-Y-stable[-ee] and security/master
    4. Update the VERSION and component version files :white_check_mark:
    5. Push security/X-Y-stable[-ee] and security/master to the Security and Build repositories.

    Publication

    Security releases pull from Build using the security/X-Y-stable or security/X-Y-auto-deploy-YYYYMMMDD branch, depending on what we're releasing.

    After publication, the security/* branches (including security/master) are merged into their normal counterparts (e.g., security/12-3-stable gets pulled into 12-3-stable) and pushed to the Canonical repository, bringing all three repositories back in sync.

    :warning: SyncRemoteService needs to be updated for the security/ prefix.

    graph LR
      subgraph Canonical
        c-master[master]
        c-stable[12-3-stable]
        c-auto-deploy[12-3-auto-deploy-20190911]
      end
    
      subgraph Build
        bs-master[security/master]
        bs-stable[security/12-3-stable]
        bs-auto-deploy[security/12-3-auto-deploy-20190911]
      end
    
      bs-stable -->|publish| c-stable
      bs-master -->|publish| c-master
      bs-auto-deploy -->|publish| c-auto-deploy

    All tags are synced from Build to Canonical.

    graph LR
      subgraph Canonical
        c-tag[v12.3.2]
      end
    
      subgraph Build
        bs-tag[v12.3.2]
      end
    
      bs-tag -->|publish| c-tag

    Components

    These are the projects that make up the GitLab product.

    Tooling

    These projects orchestrate a release of the GitLab product.

    Nothing to do

    • gitlab-elasticsearch-indexer
      • No mentions of dev.gitlab.org
    • gitlab-monitor
      • None found
    • gitlab-shell
      • No mentions of dev.gitlab.org

    Process changes

    Backup 2- Development workflow, not germane to discussion here

    Day-to-day development workflow

    :white_check_mark: Nothing changes! :tada:

    Features and bug fixes get developed in Canonical, merging to the master branch.

    If a merge request needs to be included in the current auto-deploy branch, it receives the Pick into auto-deploy label, which we cherry-pick on a regular schedule.

    If a merge request needs go into a specific stable branch to be included in a patch release, it receives the Pick into X.Y label, which we cherry-pick during the patch release process.

    graph LR
      subgraph Canonical
        c-master(master)
        c-stable(12-3-stable)
        c-auto-deploy(12-3-auto-deploy-20190911)
        c-feature(feature/some-new-feature)
        c-bug(15015-some-bug-fix)
    
        c-feature --> c-master
        c-bug --> c-master
        c-bug -.->|Pick into 12.3| c-stable
        c-bug -.->|Pick into auto-deploy| c-auto-deploy
      end

    Security development workflow

    A merge request should be opened in Security, referencing the confidential vulnerability issue from Canonical, targeting the security/master branch.

    The security/master merge request goes through review and, when approved, gets reassigned to the @gitlab-release-tools-bot for merging.

    graph TD
      subgraph Security
        s-master(security/master)
    
        s-fix-master(security/15015-fix-mermaid-xss)
    
        s-fix-master --> s-master
      end
    
      subgraph Canonical
        c-issue((#15015 XSS in Mermaid))
      end
    
      s-fix-master -.-> c-issue

    Once the security/master merge request is reviewed, approved, and assigned to the bot, backport merge requests can be opened in Security targeting the previous three stable branches. These backports go through a simplified review process and then get assigned to the @gitlab-release-tools-bot user for merging.

    graph TD
      subgraph Security
        s-fix-12-2(security/15015-fix-mermaid-xss-12-2)
        s-fix-12-1(security/15015-fix-mermaid-xss-12-1)
        s-fix-12-0(security/15015-fix-mermaid-xss-12-0)
    
        s-stable-12-2(security/12-2-stable)
        s-stable-12-1(security/12-1-stable)
        s-stable-12-0(security/12-0-stable)
    
        s-fix-12-2 --> s-stable-12-2
        s-fix-12-1 --> s-stable-12-1
        s-fix-12-0 --> s-stable-12-0
      end
    
      subgraph Canonical
       c-issue((#15015 XSS in Mermaid))
      end
    
      s-fix-12-2 -.-> c-issue
      s-fix-12-1 -.-> c-issue
      s-fix-12-0 -.-> c-issue

    Because all branches in Security are prefixed with security/, security merge requests can be merged as soon as they are deemed ready.

    Security MR validation

    0 of 6 checklist items completed · Edited by Mayra Cabrera

    Linked items 0

  • Link items together to show that they're related or that one is blocking others.

    Activity

    • All activity
    • Comments only
    • History only
    • Newest first
    • Oldest first