Test Plan for Add member counts to GitLab.com billing areas

Test Plan

Introduction

This test plan is for #7772 (closed) / #7438 (closed). This feature adds subscription table to GitLab.com billing areas

Scope

  • Billing is only available on gitlab.com
  • Billing is at top level group. All members of sub-groups are taken into account.
  • Includes separate calculation logic for Free, Trail and Paid plans.
  • This test plan only covers the subscription table. Seat purchase history and Pipeline minutes are out of scope.

ACC Matrix

The matrix below identifies the Attributes, Components, and Capabilities relevant to the scope of this test plan.

Attributes (columns) are adverbs or adjectives that describe (at a high level) the qualities testing is meant to ensure Components have.

Components (rows) are nouns that define major parts of the product being tested.

Capabilities link Attributes and Components. They are what your product needs to do to make sure a Component fulfils an Attribute

This feature includes the "Admin" functional area and so they included in the matrix.

For more information see the Google Testing Blog article about the 10 minute test plan and this wiki page from an open-source tool that implements the ACC model.

The numbers indicate the count of Capabilities at each intersection of Attribute and Component

Simple Secure Responsive Obvious Stable
Admin 2 1 4
API 2 1

Capabilities

  • Admin (billing) is

    • Simple
      • Current status of subscription is clear with "Seats in subscription" and "Seats in use".
      • Any overage is clearly communicated with Max seats used and Seat owed.
    • Stable
      • Blocked members do not count towards Seats in use.
      • Guest members do not count towards "Seats in subscription" and "Seats in use" in Gold plan.
      • Guest members count towards "Seats in subscription" and "Seats in use" in Gold plan.
      • All members of the subgroups are taken in to account.
    • Responsive
      • The subscription table is responsive

  • API is

    • Secure
      • It does not expose other user's subscriptions
      • It does not expose groups subscriptions of the user is not a member
    • Obvious
      • It easy to understand the purpose of the endpoint

Test Cases

Capabilities mentioned above can be used to guide the testing. Some cases not completely obvious from the capabilities are mentioned in below list. This list, however, should not be considered exhaustive and should only be used as a reference point for actual tests.

When adding new automated tests, please keep testing levels in mind.

We are likely to not need e2e QA tests for the below test cases as the would be easily covered at lower levels.

Paid plan (Gold, Silver or Bronze):

Scenario 1: "Seats currently in use" < "Seats in subscription" and "Max seats used" <= "Seats in subscription"

  • "Seats owed" should equal 0

Scenario 2: "Seats currently in use" == "Seats in subscription" and "Max seats used" == "Seats in subscription"

  • "Seats owed" should equal 0

Scenario 3: "Seats currently in use" > "Seats in subscription" and "Max seats used" > "Seats in subscription"

  • "Seats owed" should equal "Max seats used" - "Seats in subscription"

Scenario 4: "Seats currently in use" < "Seats in subscription" and "Max seats used" > "Seats in subscription"

  • "Seats owed" should equal "Max seats used" - "Seats in subscription"

Scenario 5: Add a guest user to the Gold plan

  • "Seats currently in use", "Max seats used" and "Seats owed" do not change

Scenario 6: Add a guest user to the Silver plan

  • "Seats currently in use", "Max seats used" and "Seats owed" update accordingly.

Trial Plan:

Scenario 1: Add a user to group:

  • "Seats in subscription" should equal "Max seats used" and should increment with each user added.
  • "Seats currently in use" should equal the number of users in group.
  • "Seats owed" should equal 0

Scenario 2: Remove a user from group:

  • "Seats in subscription" should equal "Max seats used" and should not change with each user removed.
  • "Seats currently in use" should equal the number of users in group.
  • "Seats owed" should equal 0

Free Plan:

Scenario 1: Add a user to group:

  • "Seats in subscription" should equal "Max seats used" and should increment with each user added.
  • "Seats currently in use" should equal the number of users in group.
  • "Seats owed" should equal 0

Scenario 2: Remove a user from group:

  • "Seats in subscription" should equal "Max seats used" and should not change with each user removed.
  • "Seats currently in use" should equal the number of users in group.
  • "Seats owed" should equal 0
Edited by Sanad Liaquat