Update & clarify the Growth experimentation toolset

To-dos

  • Figure out the standard tooling set
    • Tooling should support and simplify adherence to the process
    • Need to research performance concerns of using Unleash on .com
    • Both Unleash & Flipper have UIs that can be used to view existing experiments & adjust their rollout strategies & targets (e.g. users, groups, etc.)
    • Unleash has variants built in, Flipper does not appear to natively support variants
    • Unleash server is node module, whereas Flipper is Ruby gem
    • Using Flipper’s built-in %-of-actors makes sense once we know we’ll have a logged in user
    • Using Flipper’s built-in %-of-time and converting that to individual users via a cookie (i.e. as in the Gitlab::Experimentation framework) makes good sense when we’re not sure we’ll have an authenticated user or when we know that we won’t have an authenticated user (i.e. sign-up & start-a-trial flows)
    • It would be nice if PMs could directly control the rollout timing & strategy of experiments

Alternative Solutions

Edited by Dallas Reedy