Skip to content

This adds logic around enabling an experiment

Jeremy Jackson requested to merge jj-build-out-enabled-behavior into master

Part of https://gitlab.com/gitlab-org/gitlab-experiment/-/issues/19.

It was already in place, but wasn’t utilized until now. Now that we understand this more, we integrated the concept.

The intended use is basically to be able to override it easily per experiment, and within the ApplicationExperiment in gitlab we may take a couple approaches to this. I'm not sure yet, but here are some options of the usage expectation:

The best performing approach might be to simply not include the feature flag yaml until you're ready to launch the experiment. I haven't tested this idea out all the way, but it's probably the best performance. This would basically optimize for scenarios in FOSS, where we could implement the experiment feature flag in EE, which would mean that if that feature flag isn't defined in FOSS we could immediately move on knowing that we should render the control and do no tracking.

class ApplicationExperiment < Gitlab::Experiment
  def enabled?
    Feature::Definition.get(name).nil?
  end
end

The downside to that approach is that as soon as you added the feature flag yaml, you would start getting control events as soon as it's deployed to production.

The alternative isn't as performant, but it may give us better timing in terms of when we roll it out and when we start getting control events. This would ultimately do one additional feature check to load the Flipper::Feature and check the state.

class ApplicationExperiment < Gitlab::Experiment
  def enabled?
    Gitlab::dev_env_or_com? && Feature.get(name).state != :off
  end
end

When experimenting in development, I realized this:

Feature.get(:new_project_readme).state # => :off
Feature.enable(:new_project_readme)
Feature.get(:new_project_readme).state # => :on
Feature.disable(:new_project_readme)
Feature.get(:new_project_readme).state # => :off
Feature.get(:new_project_readme).enable_percentage_of_time(50)
Feature.get(:new_project_readme).state # => :conditional

So as long as you haven't rolled the feature flag out at all, it's considered :off. The same is true if the feature flag isn't known, which would also mean excluding the feature flag yaml would keep it from being considered enabled -- though, I think the chatop commands can create one at any time still, which would probably cause an exception further down the chain.

I'm thinking on what gitlab should implement here.

Gitlab should also switch to using the should_track? method as well.

Edited by Jeremy Jackson

Merge request reports