E2E Test Audit - Govern
As per https://gitlab.com/gitlab-com/gitlab-OKRs/-/work_items/2543
Govern
| Spec | Basic Workflow | Shift Left? | Merge? |
|---|---|---|---|
| Govern compliance dashboard with separation of duties in an MR when there are two approvals but one of the approvers is the author shows only "author approved merge request" and "approved by committer" violations | MR set up View Compliance Dashboard View violations |
No - customer escalation regression test | |
| Govern compliance dashboard with separation of duties in an MR when there is only one approval from a user other than the author shows only "less than two approvers" violation | MR set up View Compliance Dashboard View violations |
Yes - ee/spec/frontend/compliance_dashboard/components/violations_report/report_spec.js could be expanded | |
| Govern Compliance Framework Report shows the compliance framework for each project | Apply compliance framework per project for 2 projects View compliance framework report |
Yes - ee/spec/frontend/compliance_dashboard/components/frameworks_report/report_spec.js - expand to have >1 project framework | |
| Govern Group for add and remove project access behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for add group behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for add user, change access level, remove user behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for change project creation level behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for change repository size limit behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for disable and Enable LFS behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for enable and disable 2FA requirement behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for enable and disable allow user request access behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for enable and disable membership lock behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Group for update group name behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Instance for add and delete email behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Instance for add SSH key behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Instance for change password behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Instance for failed sign in behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Instance for start and stop user impersonation behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Instance for successful sign in behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Policies List page can load Policies page and view the policies list | Load policies page | Yes - ee/spec/frontend/security_orchestration/components/policy_editor/policy_selection_spec.js covers this | |
| Govern Policies List page can navigate to Policy Editor page | Load policies page, navigate to editor | Yes - ee/spec/frontend/security_orchestration/components/policy_editor/policy_selection_spec.js should cover this, or extend | |
| Govern Project for add deploy key behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Project for add project behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Project for add user access as guest behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Project for change visibility behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Project for export file download behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern Project for project archive and unarchive behaves like audit event logs audit events for UI operations | Perform action Confirm action shown as an audit event |
Audit frontend specs exist, and audit event specs exist, but no E2E workflow | Tests already tightly coupled, possibly perform all actions up front and visit + search audit page once, searching for all events |
| Govern project vulnerability report can successfully bulk change status in vulnerability report | Set up vulnerabilities View report Change status |
No - because E2E tests backend integration with vulnerabilities state change (GraphQl call is made) | |
| Govern project vulnerability report can successfully change status of a vulnerability in vulnerability details page | Set up vulnerabilities View report Change status |
No - because E2E tests backend integration with vulnerabilities state change (GraphQl call is made) | |
| Govern Project vulnerability report validates "fix a vulnerability" workflow | Set up vulnerabilities in different states View state |
No - It is an E2E workflow | |
| Govern Scan result policy requires approval when a pipeline report has findings matching the scan result policy | Set policy Run pipeline Pipeline report |
Probably not, lends itself to an E2E workflow | Seems standalone functionality |
| Govern Security Dashboard in a Project creates an issue from vulnerability details | Populate project with vulnerabilities Create issue from vulnerability |
Probably not, lends itself to an E2E workflow | Yes merge with shows vulnerability details |
| Govern Security Dashboard in a Project shows vulnerability details | Populate project with vulnerabilities View vulnerability details |
Possibly though could be merged with create issue | Yes merge with create issue |
| Govern Security Reports dependency list has empty state | Create basic project View empty dependency list |
Yes - covered by ee/spec/frontend/dependencies/components/app_spec.js | |
| Govern Security Reports displays false positives for the vulnerabilities | Create project Populate dependencies View dependency list View false positive |
Probably not, lends itself to an E2E workflow | |
| Govern Security Reports displays security reports in the group security dashboard | Create project Populate dependencies View group security dashboard |
Probably not, lends itself to an E2E workflow | Yes merge group/pipeline/project |
| Govern Security Reports displays security reports in the pipeline | Create project Populate dependencies View pipeline |
Probably not, lends itself to an E2E workflow | Yes merge group/pipeline/project |
| Govern Security Reports displays security reports in the project security dashboard | Create project Populate dependencies View project security dashboard |
Probably not, lends itself to an E2E workflow | Yes merge group/pipeline/project |
| Govern Security Reports displays the Dependency List | Create project Populate dependencies View dependency list |
Yes, as this E2E spec does a basic count Covered off by ee/spec/frontend/dependencies/components/dependencies_table_spec.js |
|
| Govern Security Reports in a Merge Request Widget displays vulnerabilities in merge request widget | Create project Populate vulnerabilities in an MR View MR widget |
Probably not, is a holistic MR view | Yes merge with Vulnerability Management in a merge request tests |
| Govern Vulnerability management in a merge request can create an auto-remediation MR from mr security widget | Create project Populate vulnerabilities in an MR View MR widget Create auto-remediation MR |
Probably not, lends itself to an E2E workflow | If possible, merge MR widget tests |
| Govern Vulnerability management in a merge request can create an issue from a security finding in pipeline security tab | Create project Populate vulnerabilities in an MR View pipeline security tab Create issue |
Probably not, lends itself to an E2E workflow | Merge with Govern Security Reports displays security reports in the pipeline |
| Govern Vulnerability management in a merge request can create an issue from a vulnerability from mr security widget | Create project Populate vulnerabilities in an MR View MR widget Create issue |
Probably not, lends itself to an E2E workflow | If possible, merge MR widget tests |
| Govern Vulnerability management in a merge request can dismiss a security finding with reason from pipeline security tab | Create project Populate vulnerabilities in an MR View pipeline security tab Dismiss |
Probably not, lends itself to an E2E workflow | Merge pipeline security tab tests |
| Govern Vulnerability management in a merge request can dismiss a vulnerability with a reason from mr security widget | Create project Populate vulnerabilities in an MR View MR widget Dismiss |
Probably not, lends itself to an E2E workflow | If possible, merge MR widget tests |
| Govern Vulnerability report in a project can export vulnerability report to csv | Populate project with vulnerabilities View security dashboard Export to CSV Validate CSV |
Probably not ee/spec/frontend/security_dashboard/components/shared/vulnerability_report/vulnerability_report_header_spec.js has a test for the button but no validation of CSV |
If possible merge with other Security Dashboard tests |
| Govern vulnerability report with jira integration can successfully create a JIRA issue from vulnerability details page | Populate project with vulnerabilities View vulnerability details Create JIRA issue |
Probably not, is an example of an E2E third party integration | If possible merge with other Vulnerability Details tests |