Skip to content
Snippets Groups Projects

Use project bot users to run scan execution policies pipelines

Merged Andy Schoenen requested to merge andysoiron/security-scan-results-policy-bots into master

What does this MR do and why?

This MR adds a:

  • Feature flag scan_execution_bot_users.
  • Column bot_user_id to security_orchestration_policy_configurations.
  • Bot user type security_policy_bot.
  • Worker to create bot users and add them as guest to a project.
  • Change to run the worker whenever a new Security::OrchestrationConfiguration is assigned.

With the Feature flag scan_execution_bot_users enabled. With every new security configuration is assigned to a project, it should create a bot user and add it as a guest to the project.

In the next iteration, we want to use the bot user to trigger scan execution pipelines. Currently, the last user that edited the security policy project is used as a triggerer for the pipelines.

Related issue #394958 (closed)

Screenshots or screen recordings

Screenrecording coming soon.

How to set up and validate locally

There are 4 cases to validate:

Preparation

  1. Switch to the andysoiron/security-scan-results-policy-bots branch
  2. Run migrations rails db:migrate
  3. Restart GDK
  4. Enable the feature flag Feature.enable(:scan_execution_bot_users)
  5. Create a new project.

1. Assign a security policy project

  1. On the right sidebar, select Security and Compliance* and Policies.
  2. Select Edit policy project.
  3. Choose any project and select Save.
  4. On the right sidebar, select Project information* and Members.
  5. There should now be a bot member named GitLab Security Policy Bot.

2. Change the security policies project

  1. On the right sidebar, select Security and Compliance* and Policies.
  2. Select Edit policy project.
  3. Choose any project other and select Save.
  4. On the right sidebar, select Project information* and Members.
  5. The bot user should still be there GitLab Security Policy Bot.
  6. Select the username to get to the user page.
  7. Remember the user ID
  8. Find your project ID and copy it.
  9. Look up the Security::OrchestrationPolicyConfiguration for the project
    Project.find(YOUR_PROJECT_ID).security_orchestration_policy_configuration
  10. The bot_user_id field should match the ID of the bot user.

3. Remove the security policies project

  1. On the right sidebar, select Security and Compliance* and Policies.
  2. Select Edit policy project.
  3. Select the trash can symbol next to the security policies project name.
  4. Select Save.
  5. On the right sidebar, select Project information* and Members.
  6. The GitLab Security Policy Bot user should be removed.

4. Remove a bot user member

  1. On the right sidebar, select Security and Compliance* and Policies.
  2. Select Edit policy project.
  3. Choose any project and select Save.
  4. On the right sidebar, select Project information* and Members.
  5. There should now be a bot member named GitLab Security Policy Bot.
  6. Select the three dots (more actions) icon on the right side of the member entry.
  7. Select Remove Member.
  8. Select Remove Member from the popup window again.
  9. Find your project ID and copy it.
  10. Look up the Security::OrchestrationPolicyConfiguration for the project
    Project.find(YOUR_PROJECT_ID).security_orchestration_policy_configuration
  11. The bot_user_id field should be nil.

Database

Rollback migrations:

rails db:migrate:down:main VERSION=20230419192748
rails db:migrate:down:ci VERSION=20230419192748
rails db:migrate:down:main VERSION=20230419193807
rails db:migrate:down:ci VERSION=20230419193807
rails db:migrate:down:main VERSION=20230425124907
rails db:migrate:down:ci VERSION=20230419193807

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Edited by Andy Schoenen

Merge request reports

Loading
Loading

Activity

Filter activity
  • Approvals
  • Assignees & reviewers
  • Comments (from bots)
  • Comments (from users)
  • Commits & branches
  • Edits
  • Labels
  • Lock status
  • Mentions
  • Merge request status
  • Tracking
  • Contributor
    1 Warning
    :warning:

    featureaddition and featureenhancement merge requests normally have a documentation change. Consider adding a documentation update or confirming the documentation plan with the Technical Writer counterpart.

    For more information, see:

    1 Message
    :book: This merge request adds or changes files that require a review from the Database team.

    This merge request requires a database review. To make sure these changes are reviewed, take the following steps:

    1. Ensure the merge request has database and databasereview pending labels. If the merge request modifies database files, Danger will do this for you.

    2. Prepare your MR for database review according to the docs.

    3. Assign and mention the database reviewer suggested by Reviewer Roulette.

    4. Kick off the db:gitlabcom-database-testing manual job. This job can also be used before requesting review to test your migrations against production data.

    The following files require a review from the Database team:

    • db/migrate/20230419192748_add_bot_user_id_to_security_orchestration_policy_configurations.rb
    • db/migrate/20230419193807_add_foreign_key_for_bot_user_id_to_security_orchestration_policy_configurations.rb
    • db/schema_migrations/20230419192748
    • db/schema_migrations/20230419193807
    • db/structure.sql

    Reviewer roulette

    Changes that require review have been detected!

    Please refer to the table below for assigning reviewers and maintainers suggested by Danger in the specified category:

    Category Reviewer Maintainer
    backend Madelein van Niekerk current availability (@maddievn) (UTC+8, 6 hours ahead of @Andysoiron) Jan Provaznik current availability (@jprovaznik) (UTC+2, same timezone as @Andysoiron)
    database Vitali Tatarintev current availability (@ck3g) (UTC+2, same timezone as @Andysoiron) Michał Zając current availability (@Quintasan) (UTC+2, same timezone as @Andysoiron)
    ~"migration" No reviewer available No maintainer available
    ~"Threat Insights backend" Reviewer review is optional for ~"Threat Insights backend" Zamir Martins Filho current availability (@zmartins) (UTC+0, 2 hours behind @Andysoiron)

    To spread load more evenly across eligible reviewers, Danger has picked a candidate for each review slot, based on their timezone. Feel free to override these selections if you think someone else would be better-suited or use the GitLab Review Workload Dashboard to find other available reviewers.

    To read more on how to use the reviewer roulette, please take a look at the Engineering workflow and code review guidelines. Please consider assigning a reviewer or maintainer who is a domain expert in the area of the merge request.

    Once you've decided who will review this merge request, assign them as a reviewer! Danger does not automatically notify them for you.

    Sidekiq queue changes

    This merge request contains changes to Sidekiq queues. Please follow the documentation on changing a queue's urgency.

    These queues were added:

    • security_orchestration_configuration_create_bot

    If needed, you can retry the :repeat: danger-review job that generated this comment.

    Generated by :no_entry_sign: Danger

  • A deleted user added Data WarehouseImpact Check label
  • Andy Schoenen changed the description

    changed the description

  • Andy Schoenen added 1 commit

    added 1 commit

    • e9ab0a22 - Add foreign_key for bot_user_id

    Compare with previous version

  • Andy Schoenen added 1 commit

    added 1 commit

    Compare with previous version

  • Contributor

    Allure report

    allure-report-publisher generated test report!

    e2e-package-and-test: :x: test report for 290436d9

    expand test summary
    +-----------------------------------------------------------------------+
    |                            suites summary                             |
    +------------------+--------+--------+---------+-------+-------+--------+
    |                  | passed | failed | skipped | flaky | total | result |
    +------------------+--------+--------+---------+-------+-------+--------+
    | Create           | 292    | 0      | 42      | 50    | 334   | ❗     |
    | Configure        | 0      | 0      | 6       | 0     | 6     | ➖     |
    | Plan             | 120    | 0      | 0       | 66    | 120   | ❗     |
    | Verify           | 108    | 0      | 8       | 88    | 116   | ❗     |
    | Govern           | 92     | 0      | 0       | 92    | 92    | ❗     |
    | Data Stores      | 68     | 0      | 0       | 16    | 68    | ❗     |
    | Manage           | 64     | 2      | 6       | 16    | 72    | ❌     |
    | Secure           | 14     | 0      | 10      | 14    | 24    | ❗     |
    | Release          | 12     | 0      | 0       | 8     | 12    | ❗     |
    | Analytics        | 4      | 0      | 0       | 4     | 4     | ❗     |
    | Package          | 0      | 0      | 6       | 0     | 6     | ➖     |
    | Monitor          | 16     | 0      | 4       | 16    | 20    | ❗     |
    | Framework sanity | 0      | 0      | 2       | 0     | 2     | ➖     |
    | Fulfillment      | 4      | 0      | 48      | 0     | 52    | ✅     |
    | ModelOps         | 0      | 0      | 2       | 0     | 2     | ➖     |
    | Growth           | 0      | 0      | 4       | 0     | 4     | ➖     |
    +------------------+--------+--------+---------+-------+-------+--------+
    | Total            | 794    | 2      | 138     | 370   | 934   | ❌     |
    +------------------+--------+--------+---------+-------+-------+--------+
  • Andy Schoenen changed the description

    changed the description

  • Andy Schoenen added 1 commit

    added 1 commit

    Compare with previous version

  • Andy Schoenen marked the checklist item Remove bot user on unassign as completed

    marked the checklist item Remove bot user on unassign as completed

  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Loading
  • Please register or sign in to reply
    Loading