## Summary For successful launch of the [Zuora Orders Harmonization (OH) Project](https://gitlab.com/gitlab-com/business-technology/enterprise-apps/intake/-/issues/158), we need to identify and test end to end user flows related to Orders. This involves flows originating from Web Direct sales in CustomersDot, as well as those from sales-assisted deals. For the latter, we will be leaning on the ~"Enterprise Applications" to create test data for sales-assisted flow. Details of the project, timeline and additional [background information](https://gitlab.com/groups/gitlab-org/-/epics/6411) can be found in the mother epic and referenced issues. ### Proposal * [x] Identify all test scenarios and document them in separate issues. * [x] Create a testing table with status and Tester columns. * [x] Describe test plan and how to document test results. * [x] Execute test plan after OH is enabled in Zuora Central Sandbox. ### Result Any pending bugs that were not caught through unit test or integration test are identified and documented. ## E2E Testing ### Use Case Scenarios We should walk through the testing steps for each of the follow use case scenarios. Note the testing steps and location to record Pass/Fail is listed in the related issue. | Use Case ID | Product | Description | |-------------|---------|-------------| | [UC-1](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3789) | Self-managed | Customer purchases a new SM subscription | | [UC-2](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3790) | SaaS | Customer purchases a new SaaS subscription | | [UC-3](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3791) | SaaS | Customer purchases an Add-on subscription | | [UC-4](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3792) | Self-managed | Customer renews an existing SM subscription | | [UC-5](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3793) | SaaS | Customer renews an existing SaaS subscription | | [UC-6](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3796) | Self-managed | Customer upgrades an existing SM subscription | | [UC-7](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3794) | SaaS | Customer upgrades an existing SaaS subscription | | [UC-8](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3798) | Self-managed | SM Subscription created through sales-assisted order | | [UC-9](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3799) | SaaS | SaaS Subscription created through sales-assisted order | | [UC-10](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3821) | Any | Customer renews an existing EoA subscription | | [UC-11](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3822) | Self-managed | Sales-assisted subscription cancellation | | [UC-12](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3824) | Any | Quarterly reconciliation adds seats to a subscription | ## Testing Plan 1. The first testing sessions will take place in customers.staging.gitlab.com once Orders Harmonization is enabled in Zuora Central Sandbox and the `orders_harmonization` feature flag is enabled. 1. Ideally we should have at least 2 testers testing each scenario. 1. Test results should be [documented](https://gitlab.com/groups/gitlab-org/-/epics/6606#documenting-testing-results) in the individual testing issue for the applicable scenario. 1. Where possible, [record your tests](https://gitlab.com/groups/gitlab-org/-/epics/6606#recording-tests). ## Testing Team Please feel free to assign the testing issue you've volunteered for to yourself since each issue can have multiple assignees. Update the table below with the testing status (see legend below) as you progress through your use case. | Use Case ID | Tester 1 | Tester 2 | |-------------|--------------|----------------| | [UC-1](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3789) | :white_check_mark: @tyleramos | | | [UC-2](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3790) | :white_check_mark: @tyleramos | | | [UC-3](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3791) | :white_check_mark: @tyleramos | | | [UC-4](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3792) | :white_check_mark: @tyleramos | | | [UC-5](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3793) | :white_check_mark: @tyleramos | | | [UC-6](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3796) | :white_check_mark: @tyleramos | | | [UC-7](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3794) | :white_check_mark: @tyleramos | | | [UC-8](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3798) | :white_check_mark: @tyleramos | :white_check_mark: @a_luna | | [UC-9](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3799) | :white_check_mark: @tyleramos | | | [UC-10](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3821) | :white_check_mark: @tyleramos | | | [UC-11](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3822) | :white_check_mark: @a_luna | | | [UC-12](https://gitlab.com/gitlab-org/customers-gitlab-com/-/issues/3824) | :white_check_mark: @a_luna | | <details><summary>Testing Status Legend</summary> | Emoji | Status | | ------ | ------ | | :white_check_mark: | Completed & Passed | | :x: | Completed with some failures | | :no_entry: | Blocked | | :hourglass: | In progress | | :white_large_square: | Not started | </details> ## Recording Tests While testing, it is recommended to self-record walking through the (testable) testing steps. Tips for making this beneficial for others: 1. Create short, specific test videos. Don't record a 3-hour testing session. This makes playback for purposes of investigating issues difficult. Instead, create short, specific testing videos. 1. Name your videos after the specific test case for ease of reference. 1. Save your testing videos in the [Orders Harmonization Testing YouTube playlist](https://www.youtube.com/playlist?list=PL05JrBw4t0KpXU6KqwLuXDAGwMDxlVrLM). 1. Make sure your video is Private visibility. 1. Link to your video in your test results comment ## Documenting Testing Results Because we will have multiple testers for each scenario, each tester should create a single (running) comment in the specific testing issue, with their results. Utilize the following template for your comment: | Step ID | Video |Comments | Pass/Fail | Issues Created w/ Priority| |---|---|---|---|---| |T-7|[link to youtube video]|Followed steps as described | Fail | `#12345` - P1 <br> <br> `#98765` - P2| ### Creating Issues Please make sure your issues contain: 1. A summary of the problem 1. Steps to reproduce or link to video reproduction (videos are really helpful!) 1. Appropriate labels to signal priority of the issue ## Other Artifacts - [Ent Apps Standard Test Cases](https://docs.google.com/spreadsheets/d/1fjig4Sg37tJmKr4KODFfRDt91h6FYsSltTGBb7deKo4/edit#gid=0) - [Example E2E Test Plan from SuperSonics](https://gitlab.com/groups/gitlab-org/-/epics/6217) - [Another example of E2E Test Plan from Vincy](https://gitlab.com/gitlab-org/fulfillment-meta/-/issues/246)