Skip to content
Snippets Groups Projects

Document user workflows for Secret Detection

    • Open Issue created by Amar Patel @amarpatel

    Problem to solve

    We would like to document the user workflows our group is responsible for. By documenting these workflows, we’ll be able to identify any gaps in test coverage and work with the Test Engineering Team in order to get automated testing set up to cover those gaps. In addition to this, I see this contributing to our efficiency by giving our support engineers a place to reference for how our workflows are expected to work, therefore reducing the number of requests for help we get.

    Proposal

    We should document all customer-facing workflows that groupsecret detection is responsible for. Then collaborate with Test Engineering Team to ensure there is an E2E test set up for each workflow/scenario, or come up with a plan to address the gaps.

    How we measure success

    • Identify all customer facing workflows/journeys
    • E2E tests exist for all workflows
    • Decide on how we want to capture this data (standard format, spreadsheet, etc.)
    • Create a repository of example projects that we can use for manually testing these workflows when necessary
    • Document a process for ensuring E2E tests are created for any new workflow/feature we create.

    Known workflows (WIP)

    The scenarios listed below are what we used to test updating the latest CI/CD Template to stable in [Spike] Determine if latest secret detection CI... (#509772 - closed) • Vishwa Bhat • 17.9 • On track. Workflow diagram forthcoming.

    Scenario 1: The latest CI/CD template is chosen for the Scan Execution Policy

    • When .gitlab-ci.yml file is NOT present in the project (SEP internally creates one but won't be visible in the project)
      • A branch pipeline is created for every commit to a branch. However, when an MR is created for that branch, neither the MR pipeline nor the Branch pipeline is triggered. This behavior causes MR to be mergeable effectively bypassing SEP. :x:
      • However, when using the stable SD CI template, a branch pipeline is created for every commit regardless of whether an MR is created for that branch. Note that the MR security widget shows only the latest branch pipeline result due to the same situation present for other scenarios. :question:
    • When .gitlab-ci.yml file is already present in the project
      • A branch pipeline is created for every commit to a branch. When an MR is created for that branch, the MR pipeline is triggered.
      • The vulnerability Report and MR security widget behave as expected for all the test scenarios followed when using latest template without SEP. :white_check_mark:
    • Sample projects used for testing: Policy Project | Test Project

    Conclusion: When .gitlab-ci.yml file is already present in the project, all the test scenarios worked as expected :white_check_mark:. However, when .gitlab-ci.yml file is NOT present in the project, the MR pipeline is NOT created when the policy is created with the latest SD template :x:. I didn't dig into this further as it was not in the scope of the spike, I could look at it after completing this spike task.

    Scenario 2: Using latest CI/CD template for a new project

    • Scenarios tested:
      • Introduce a secret in a commit before creating MR to trigger the Branch pipeline. Verify if the analyzer detects the secret in the vulnerability report :white_check_mark:
      • Introduce a secret in a commit in a new branch and then Create MR. Verify if the MR widget shows the existing secret finding that was introduced before creating MR :white_check_mark:
      • Introduce a secret in a commit after creating MR to trigger the MR pipeline. Verify if the analyzer detects the secret and the MR widget shows the secret finding :white_check_mark:
      • Introduce a secret in the middle of the commit chain in the source branch after creating MR. Verify if the analyzer detects the secret and the MR widget shows the secret finding :white_check_mark:
    • Sample project used for testing: link

    Conclusion: For a customer who is starting a new project without Scan Execution Policies or ADO, the latest template change works as expected for the above test scenarios :white_check_mark:

    Note: In case you have any scenario in mind that I missed to consider, feel free to add it as a comment under this post. I'll run the test in that scenario and post the results.

    Scenario 3: Migrating from existing stable template to latest template (without SEP/ADO)

    • Currently, the stable template isn't configured to run the SD analyzer in MR pipelines. Due to the missing MR pipeline, the MR security widget ends up showing the source branch's latest branch pipeline results in the widget. This confused some customers thinking the secret findings detected for the initial commit later "disappeared" after making subsequent secretless commits in the MR widget.
    • After switching to the latest template:
      • For a new branch, every commit pushed resulted in triggering a Branch pipeline. If a commit contained a secret, the analyzer included the secret finding in the pipeline's vulnerability report.
      • When an MR is created for a branch, for every commit pushed to the source branch, an MR pipeline is triggered instead of a Branch pipeline. The analyzer ran the scan on all the commits of the source branch instead of choosing only the latest commit (as it happened for the stable template). This resulted in including all the secret findings detected across the commits in the MR pipeline's vulnerability report and thereby in the MR security widget too.
    • Sample project used for testing: link

    Conclusion: For an existing customer who is not using Scan Execution Policies or ADO, moving the latest template changes to stable template works as expected :white_check_mark:

    Scenario 4 - AutoDevOps (ADO)

    • ADO CI/CD Template link for reference.
    • ADO doesn't support the MR pipeline so it refers to stable SD template. Needless to say, changing the SD template to latest wouldn't make any difference.
    • MR Security widget considers only the latest commit's branch pipeline results instead of all the commits in the source branch. This leads to showing secret findings in the widget only if the latest commit has any secret committed in it.
    • Based on the proposal suggested in MR pipeline support in the Sec templates issue, we'll likely drop the support for MR pipelines in ADO due to implementation efforts. As per the timeline, this decision should be made soon.

    Conclusion: We might not provide MR pipeline support for Secret Detection in the projects using Auto DevOps (Decision pending).

    Edited by Alana Bellucci

    Designs

      Child items ...

      2. Activity

      Please register or sign in to reply