FY21-Q1 Quality IACV OKR: Ensure enterprise grade readiness by improving test coverage, reference architectures and test stability => 81%
Key Results
-
Key result: Design and roll out department official on-call process based on existing pipeline triage rotation. => 100%, completed
-
Key result: Close out 3 remaining enterprise end-to-end test gap coverage. https://gitlab.com/gitlab-org/quality/team-tasks/issues/368 => 30%, 3 tests in-progress
Sub OKRs
-
Key result: Enablement QE Produce AWS equivalent reference architectures and increase performance test coverage. => 100%
-
Key result: Dev QE Create a reliable suite of end-to-end tests to guard every deploy in the release process and gain insights into historical reliability. => 92%
Retrospection
Good
- Reliable tests process is useful and has helped bring clarity into the release process.
- Slack improvements to the reporting is very useful.
- We removed the key result to build the AWS reference architecture and instead scaled back to adding untested documentation. I think this is good because it aligned with GitLab values and gave breathing room to our team members during the hectic time of the global pandemic.
- We accomplished all of the other key results and completed this OKR ahead of schedule!
Bad
- We still do not have insights into historical tests reliability.
- We had some confusion early on with methodology around building the cloud native architecture, and then again later on about whether this work needed to be in the documentation and where. I think we learned that we should have touched base with Distribution sooner, because once we sat down with Jason it really took off.
- We did not make progress towards the end of the quarter due to higher priority work streams.
- Mark -> Gitaly HA tasks and few enhancements related to Testcase Management System & Customer Escalation tasks
- Tomislav -> Gitaly HA tasks
- Sanad, Desiree -> Githost working Group
Try
- We need to add more reliable tests that has a high impact value of detection on features that are used most often.
- I think we could have been a little more ambitious in setting in the Enablement OKR since we completed it early. it indicates that we could have shot for the stars a bit more. I want to temper this given the global state of affairs right now, but even with sensitivity to everyone's increased stress and distracting life factors, we may have still been able to accomplish the AWS reference architecture built-out and perhaps even more related features.
Edited by Mek Stittri