[MOVED] Dogfood GitLab feature flag implementation for Growth experiments
** Moved to gitlab-org/gitlab#321749 (closed) **
Background
- The current Growth experimentation.rb implementation uses Flipper feature flags behind the scenes (this is abstracted), allowing A/B testing (control vs experiment group)
- Some GitLab projects (CustomersDot) use unleash feature flags (Category Direction, GitLab Feature Flag feature)), a few experiments have been run using this
- Growth is building out the experiment capability for GitLab.com to meet Growth product requirements #1 (closed)
- Ops is looking at a business case for A/B testing gitlab-com/Product#1634
Dogfooding option
GitLab Feature Flag service using Unleash is a potential solution. This would involve enabling Unleash on GitLab.com for experimentation initially, as this is deemed lower risk than adopting for all GitLab development feature flags. There is some work to do to enable this. Growth can act as the customer.
Experiment feature flag type using Unleash
There is a proposal to introduce a feature flag YAML type for experiments. Once Unleash is supported and enabled, this type would use the (new) Unleash adapter. If implemented in the interim (using Flipper), the plan would be to migrate to Unleash when available.
gitlab-org/gitlab#246620 (comment 434417420)
Future considerations
There is a proposal to add A/B testing to the product. Given there is no timeline for this yet, it is out of scope for Growth in the short to medium term, but should be revisited once there is plan to implement this in line with our preference for dogfooding https://about.gitlab.com/handbook/engineering/#dogfooding
The work we do here can and should be used to help drive the implementation of A/B test capabilities within the product.
Proof of concept
Example experiment running on CustomersDot using GitLab Feature Flag feature:
The initial problem revolved around direction for GitLab Feature Flags feature - whether Unleash and/or Flipper would be supported for use on GitLab.com. Documented in this comment #5 (comment 434678611), although this no longer blocks us.