Skip to content
Snippets Groups Projects
Open A/B testing based on Feature Flags
  • A/B testing based on Feature Flags

  • A/B testing based on Feature Flags

    Open Epic created by Orit Golowinski

    This page may contain information related to upcoming products, features and functionality. It is important to note that the information presented is for informational purposes only, so please do not rely on the information for purchasing or planning purposes. Just like with all projects, the items mentioned on the page are subject to change or delay, and the development, release, and timing of any products, features, or functionality remain at the sole discretion of GitLab Inc.

    Problem to solve

    A/B testing, also known as split testing, is basically an experiment where you "split" your audience to test a number of variations of possible code. In other words, you can show version A to one half of your audience, and version B to another (percentage is subject to change).

    image

    We want to be able to control deployments based on experiments defined in Gitlab.

    Intended users

    • Release managers
    • Support managers
    • Customer success managers (for beta/ test users)
    • Developers awaiting feedback to select the "right" feature path

    Internal groups:

    • Marketing
    • Growth

    Further details

    Proposal

    User should be able to define A/B testing through our Feature Flags interface image

    This feature flag should support N (TBD) variants Each variant should consist of

    1. Name
    2. Description
    3. Percentage

    UX to be provided.

    A good base for development could be using % rollout for Feature Flags which could mimic a/b testing at 50%

    For the first phase we should be able to allow traffic to go based on the percent configured. Next phase would be to select a "winner" based on metrics that will be user defined such as performance/completing an action etc. An even later phase would be to automatically direct the traffic to the "winner" after a specific user defined time or threshold is reached (# of users logged in or similar)

    Permissions and Security

    Documentation

    Testing

    What does success look like, and how can we measure that?

    Number of users using a/b testing strategy

    What is the type of buyer?

    Links / references

    Edited by 🤖 GitLab Bot 🤖

    Linked items 0

  • Link items together to show that they're related or that one is blocking others.

    Activity

    • All activity
    • Comments only
    • History only
    • Newest first
    • Oldest first