Review E2E test coverage for Govern: compliance
Context: #1631 (comment 1269164818)
This issue is to review E2E test coverage for groupcompliance and take inventory of current E2E coverage.
Lower level test coverage is being tracked here.
List of compliance features
https://about.gitlab.com/handbook/product/categories/features/#governcompliance-group
Compliance Report
- Separation of duties: E2E spec
Group Audit Events
Table below to review the test coverage.
| Group Audit Event | Test Coverage? |
|---|---|
| Group name or path changed | E2E spec |
| Group repository size limit changed. | E2E spec |
| Group created or deleted | E2E spec |
| Group changed visibility | |
| User was added to group with which permissions | E2E spec |
| User sign in using group SAML | |
| Changes to group SAML configuration | |
| Permissions changes of a user assigned to a group | E2E spec |
| Removed user from group | E2E spec |
| Project repository imported into group | |
| Project shared with group | |
| Removal of a previously shared group with a project | |
| LFS enabled or disabled | E2E spec |
| Shared runners minutes limit changed. | |
| Membership lock enabled or disabled. | E2E spec |
| 2FA enforcement or grace period changed. | E2E spec |
| Roles allowed to create project changed | |
| Group CI/CD variable | |
| Compliance framework created, updated, or deleted | |
| Event streaming destination created, updated, or deleted. | |
| Instance administrator started or stopped impersonation of a group member | |
| Group deploy token was successfully created, revoked, or deleted | |
| Failed attempt to create a group deploy token | |
| IP restrictions changed | |
| Changes to push rules | |
| MR approvals settings | |
| Changes to streaming audit destination | |
| Group had a security policy project linked, changed or unlinked | |
| An environment is protected or unprotected |
Project Audit Events
Only added covered audit events below. For the rest please check https://docs.gitlab.com/ee/administration/audit_events.html#project-events
| Project audit event | Test |
|---|---|
| Project created | E2E spec |
| User was added to the project with which permissions | E2E spec |
| Added or removed deploy keys | E2E spec |
| Project changed visibility level | E2E spec |
| Project export was downloaded | E2E spec |
| Project was archived and unarchived | E2E spec |
Chain of custody report
- Verify chain of custody report: No E2E coverage
Audit event streaming
- Verify that audit events to external streaming destination: No E2E coverage
Users and Permissions Export
- Verify that administrator can export user permissions of all users in a gitlab instance: No E2E coverage but there is lower level spec coverage
Merge request approval settings
| Configuration | Test coverage |
|---|---|
| Prevent approval by author | Limited coverage here |
| Prevent approvals by users who add commits | No E2E spec |
| Prevent editing approval rules in merge requests | No E2E spec |
| Require user password to approve | No E2E spec |
| Code Owner approval removals | No E2E spec |
There is https://gitlab.com/gitlab-org/gitlab/-/blob/master/qa/qa/specs/features/ee/browser_ui/3_create/merge_request/approval_rules_spec.rb in create stage. But it doesn't specifically check for above configuration settings
Compliance framework and compliance pipeline configuration.
| Test | E2E coverage |
|---|---|
| Create, edit and delete a compliance framework with compliance pipeline configuration for top level group | No E2E spec |
External Status Checks
- Send merge request data(create, close, change MR) to third party tools: No E2E coverage
Require an associated issue from Jira
prevent merge requests from being merged if they do not refer to a Jira issue: No E2E coverage