Quality work related to Distribution
The issue is to identity areas where QE can help to increase quality and stability working with Distribution team.
Currently identified areas
- Add initial test coverage for GitLab upgrades
- Problem: No known existing frameworks for testing GitLab upgrades. Our performance pipelines are updated mostly with nightlies and it's different from the path with which customers are updating their environments.
- Actions: Review the current test coverage (https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/1270), develop a framework/pipeline to be able to trigger upgrades and report results
- Priority: 1
- GitLab installation test coverage
- Problem: Distribution has added multiple new OS'es recently like Almalinux, AL2, etc. I believe we don't have pipelines to verify the packages for these OS'es continue to work.
- Actions: Review the current test coverage (https://gitlab.com/gitlab-org/quality/team-tasks/-/issues/882), create pipeline to trigger GitLab installation for supported OS
- Priority: 2
- Review and streamline QA pipelines in Distribution projects
- Problem: There are multiple QA pipelines, but it's not well documented what to do if something fails
- Actions: review current pipelines, explore what can be improved, how failures are tracked, document current coverage, document how to debug failures, collaborate with team if a demo is needed
- Priority: 2
-
Integration tests for Omnibus-GitLab specific functionality
- Problem: There are multiple QA pipelines, but it's not well documented what to do if something fails
- Actions: review current test coverage and what scenarios are needed to be add (pg-upgrade, automatic PG failover)
- Priority: ? (from Design thinking sessions)
- Ref Arch testing for helm charts
- Details: A set of test validation 'ref architectures' for helm charts. So we are exercising more than one deployment type, and do it in such a way that devs can deploy the same arch for their local clusters
- Priority: ? (from Design thinking sessions)
New areas
- Omnibus QA pipelines
- Problem: TBD
- TBD
Unrelated to Distribution
Areas outside of Distribution scope
- Test coverage for Backup/Restore of GitLab instances
- Problem: No existing framework for testing backup and restore as far as I'm aware.
- Explore current status and develop a continuous pipeline to test backup and restore.
- Priority: 1
- ? - not fully sure this feature is under Distribution group
Sources
- QE Distribution dashboard: https://gitlab.com/groups/gitlab-org/-/boards/2187925?label_name[]=Quality&label_name[]=devops%3A%3Asystems&label_name[]=group%3A%3Adistribution
- Distribution brainstorm: https://app.mural.co/t/gitlab2474/m/gitlab2474/1654609389941/c8ce8561287f4e1d63091e0fe364ca5916f6c3fa?sender=u3a529c23c09585bede780421
- Distribution team retrospectives