FY20-Q3 Quality IACV OKR: Improve self-managed enterprise upgrade robustness => 100%

• Key result: Define a repeatable test process to compare the performance of GitLab versions for self-managed customer upgrades with customer-facing test results | https://gitlab.com/gitlab-org/quality/team-tasks/issues/180 => 100%
• Key result: Automate 3 key performance test processes for testing multiple versions of GitLab in parallel => 100%
  • gitlab-org/quality/performance#43 (closed) | Investigate Artillery Performance
  • gitlab-org/quality/performance#49 (closed) | Add Error Reporting for Artillery Tests
  • gitlab-org/quality/performance#7 (closed) | Add success conditions to artillery load tests
• Key result: Perform performance upgrade testing of 11.9 to 12.0 => 100% (link)
• Key result: Perform performance upgrade testing of 12.0 to 12.1 => 100% (link)
• Key result: Perform performance upgrade testing of 12.1 to 12.2 => 100% (link)
• Key result: Perform performance upgrade testing of 12.2 to 12.3 => 100% (link)

Retrospective

Good

We are proud that this OKR was completed in its scaled-back form. We had a big shift in priority during the quarter (to the reference architectures) and we weren't sure if we could finish this, but we are happy we managed.

Bad

We had to scale back the OKR scope due to other priorities. We underestimated the complexity of creating these pipelines.

Try

We should build in more time for unexpected priorities to pop up. We also learned that we should not create OKRs that are driven mainly by one person. The load should be spread to make us more resilient against one person being pulled in another direction and having that impact the entire OKR. In the future, we will try to do more discovery to understand the complexities involved before settling on an estimate of time to complete.