test_plan.md 4.18 KB
Newer Older
1
# Failover Test Plan
2

3
4
## README

5
If you are volunteering, please make yourself available to join the **GCP Migration Rehearsal** call. Ping @meks for an invite.
6
7

* Zoom link: https://gitlab.zoom.us/j/859814316
8
* Slack channel: [#gcp_migration](https://gitlab.slack.com/messages/C7S4KUEPN)
9

10
11
## High-level Overview

12
This test plan requires some manual testing as well as some automated testing using [GitLab QA].
13
14
15
16
17

The automated and manual QA processes can be conducted in parallel with one another.

The plan will also involve testing during two distinct phases of the failover plan:

18
19
20
21
22
- **During the Blackout** (inside the maintenance window period):
  - All the tests conducted during this period are related to features critical to the operation of GitLab.com.
  - **The maintenance window cannot be closed until these tests pass.**
- **After the Blackout** (after the maintenance window period):
  - These tests are non-critical, but, in order to keep the maintenance window as short as possible, should be conducted after the new failover GitLab instance is public.
23
24
25
26
27

## Automated QA

### During the Blackout

28
@meks / @remy to run [GitLab QA] against failed-over environment.
29

30
31
32
- [ ] Make sure to connect to the VPN
- [ ] Make sure to export the following environment variables (you can find the
  password and tokens under the `GitLab QA` and `GitLab QA - Access tokens` 1Password items)
33

34
  ```
35
  › export GITLAB_USERNAME=gitlab-qa GITLAB_PASSWORD=xxx GITHUB_ACCESS_TOKEN=xxx
36
  ```
37

38
- [ ] Update `gitlab-qa` if needed
39

40
  ```
41
  › gem install gitlab-qa
42
  ```
43
- [ ] Automated QA completed. QA can be parallelized manually (for now):
44

45
46
47
  ```
  # Tab 1: This should take approximately 4.5 minutes

48
  › gitlab-qa Test::Instance::Staging -- qa/specs/features/api/ qa/specs/features/login/ qa/specs/features/merge_request/
49
50
51
52
53
  ```

  ```
  # Tab 2: This should take approximately 6 minutes

54
  › gitlab-qa Test::Instance::Staging -- qa/specs/features/project/
55
56
57
58
59
  ```

  ```
  # Tab 3: This should take approximately 5 minutes

60
  › gitlab-qa Test::Instance::Staging -- qa/specs/features/repository/
61
  ```
62
- [ ] Post results and failures logs + screenshots as comments of this issue
63
64
- [ ] Create `Automation Triage RELEASE_MAJOR_VERSION RC#` issues for all the
  automated QA failures and link it to this issue
Mek Stittri's avatar
Mek Stittri committed
65

66
## Manual QA
67

68
------------
69

70
PLEASE FOLLOW THESE INSTRUCTIONS AND REMOVE THEM ONCE DONE.
71

72
@meks / @remy to follow the following steps:
73

74
1. Open https://docs.google.com/spreadsheets/d/15AtBb6s2p_HvtUe5G9GUSc2ngt69X8dO-418zMuT4us/edit#gid=0
75
1. Duplicate the document and call it `GCP Migration Manual Testing - yyyy-mm-dd`
76
1. Set the permissions so that **everyone at GitLab can edit**.
77
1. Replace all the occurrences of `LINK_TO_MANUAL_TESTPLAN` below with a link to the test plan here.
78

79
------------
80

81
**Manual test plan:** LINK_TO_MANUAL_TESTPLAN
82

83
84
### During the Blackout

Rémy Coutable's avatar
Rémy Coutable committed
85
86
- [ ] When the 🔪 Chef-Runner is at the `Ensure that important processes have
   been restarted on all hosts` step, send a heads-to QA testers in `#gcp_migration`:
87

88
  ```
89
  @mkozono @bob @fran @ddavison @DylanGriffith @DaveSmith @Daniel Gruesso @toon @ruben @jedwardsjones @fabio Heads-up, manual QA testing will start soon. Please be ready (test plan: LINK_TO_MANUAL_TESTPLAN)! :)
90
91
92
93
94
  ```

- [ ] Post in `#gcp_migration`:

  ```
95
  @mkozono @bob @fran @ddavison @DylanGriffith @DaveSmith You can start performing your respective "During Blackout" manual QA scenarios on https://staging.gitlab.com. Please find the scenarios and track results at LINK_TO_MANUAL_TESTPLAN. Thanks in advance! :tada:
96
  ```
97
98
99
100
101

- [ ] Create follow-up issues for all the manual QA failures

### After the Blackout

102
- [ ] Post in `#gcp_migration` (make sure to replace `X:Y UTC` by the actual time):
103

104
  ```
105
  @ddavison @Daniel Gruesso @toon @DaveSmith @ruben @jedwardsjones @fabio You can start performing your respective "After Blackout" manual QA scenarios on https://staging.gitlab.com. The deadline for performing the "After Blackout" manual QA is set to HH:MM PM UTC. Please find the scenarios and track results at LINK_TO_MANUAL_TESTPLAN.  Thanks in advance! :tada:
106
  ```
107
108
109

- [ ] Create follow-up issues for all the manual QA failures

110
[GitLab QA]: https://gitlab.com/gitlab-org/gitlab-qa
111

112
/label ~"Failover Execution"