Audit test coverage for Scan Execution Policies
Scan Execution Policies
Unit/component
There are many and we consistently add more, so I am not going to focus on them
Integration
Most of these tests below are manual tests I do whenever I touch the code, so it will save us time to have automatic tests that do this for us
Policy List
Test scenario | Completed | Notes |
---|---|---|
See policy details in drawer | n | - |
Policy Editor
Location: spec/frontend_integration/security_orchestration/policy_editor/scan_execution_spec.js
Test scenario | Completed | Notes |
---|---|---|
Create simple Scan Execution Policy with single rule and single action and verify yaml is correct | y | Add integration tests for execution policy actions (gitlab-org/gitlab!140576 - merged) and Update integration tests (gitlab-org/gitlab!141834 - merged) |
Create simple Scan Execution Policy with single rule and single action and verify navigating to yaml mode and back does not result in a parsing error | y | Add integration tests for execution policy actions (gitlab-org/gitlab!140576 - merged) and Update integration tests (gitlab-org/gitlab!141834 - merged) |
Create complex Scan Execution Policy with multiple rules and multiple actions and verify yaml is correct | n | - |
Create complex Scan Execution Policy with multiple rules and multiple actions and verify navigating to yaml mode and back does not result in a parsing error | n | - |
Creating invalid policy in YAML mode results in parsing error in rule mode, but does not prevent user from saving | n | - |
Creating invalid policy in YAML mode results in parsing error in rule mode, but navigating back to yaml mode and updating the yaml to be valid results in an enabled parsing error | n | - |
A valid simple existing policy results in an enabled rule mode and the correct options selected | n | - |
A valid complex existing policy results in an enabled rule mode and the correct options selected | n | - |
An invalid existing policy results in a disabled rule mode | n | - |
Feature
Policy List
Test scenario | Location | Completed | Notes |
---|---|---|---|
List Scan Execution Policies for Project | ee/spec/features/projects/security/policies_list_spec.rb |
Y | Create feature tests for policies list (gitlab-org/gitlab!139024 - merged) |
List Scan Execution Policies for Group | ee/spec/features/groups/security/policies_list_spec.rb |
Y | Create feature tests for policies list (gitlab-org/gitlab!139024 - merged) |
List Scan Execution Policies for Project and Group | ee/spec/features/projects/security/policies_list_spec.rb |
Y | Won't do because the above two tests should cover everything |
Policy Editor
Test Scenarios | Location | Completed |
---|---|---|
Create new Scan Execution Policy when Security Project is not linked | ee/spec/features/projects/security/policy_editor_spec.rb |
n |
Create new Scan Execution Policy when Security Project is linked | ee/spec/features/projects/security/policy_editor_spec.rb |
n |
Delete Scan Execution Policy | ee/spec/features/projects/security/policy_editor_spec.rb |
n |
Update existing policy | ee/spec/features/projects/security/policy_editor_spec.rb |
n |
A potential policy editor feature test
describe 'scan execution policy' do
before do
sign_in(owner)
stub_licensed_features(security_orchestration_policies: true)
visit(new_project_security_policy_path(project))
find_by_testid('select-policy-scan_execution_policy').click
end
it "deletes the correct action" do
click_button _('Add action')
page.within(find_by_testid('action-1')) do
click_button _('Secret Detection')
select_listbox_item _('Container Scanning')
end
page.within(find_by_testid('action-0')) do
click_button _('Remove')
end
page.within(find_by_testid('action-0')) do
expect(page).to have_button _('Container Scanning')
end
end
it "deletes the correct condition" do
click_button _('Add condition')
page.within(find_by_testid('rule-1')) do
click_button _('specific protected branches')
select_listbox_item _('all branches')
end
page.within(find_by_testid('rule-0')) do
click_button _('Remove')
end
page.within(find_by_testid('rule-0')) do
expect(page).to have_button _('all branches')
end
end
end
Implementation Plan
-
-
Identify scenarios for tests (update the list above)
-
-
-
Identify which types of tests we want for each scenario (update the list above)
-
-
-
Verify which of these cases are completely covered by existing tests (update the list above)
-
-
-
Create the tests
-
Edited by Alexander Turinske