This adds logic around enabling an experiment
Part of https://gitlab.com/gitlab-org/gitlab-experiment/-/issues/19.
It was already in place, but wasn’t utilized until now. Now that we understand this more, we integrated the concept.
The intended use is basically to be able to override it easily per experiment, and within the ApplicationExperiment
in gitlab we may take a couple approaches to this. I'm not sure yet, but here are some options of the usage expectation:
The best performing approach might be to simply not include the feature flag yaml until you're ready to launch the experiment. I haven't tested this idea out all the way, but it's probably the best performance. This would basically optimize for scenarios in FOSS, where we could implement the experiment feature flag in EE, which would mean that if that feature flag isn't defined in FOSS we could immediately move on knowing that we should render the control and do no tracking.
class ApplicationExperiment < Gitlab::Experiment
def enabled?
Feature::Definition.get(name).nil?
end
end
The downside to that approach is that as soon as you added the feature flag yaml, you would start getting control events as soon as it's deployed to production.
The alternative isn't as performant, but it may give us better timing in terms of when we roll it out and when we start getting control events. This would ultimately do one additional feature check to load the Flipper::Feature
and check the state.
class ApplicationExperiment < Gitlab::Experiment
def enabled?
Gitlab::dev_env_or_com? && Feature.get(name).state != :off
end
end
When experimenting in development, I realized this:
Feature.get(:new_project_readme).state # => :off
Feature.enable(:new_project_readme)
Feature.get(:new_project_readme).state # => :on
Feature.disable(:new_project_readme)
Feature.get(:new_project_readme).state # => :off
Feature.get(:new_project_readme).enable_percentage_of_time(50)
Feature.get(:new_project_readme).state # => :conditional
So as long as you haven't rolled the feature flag out at all, it's considered :off
. The same is true if the feature flag isn't known, which would also mean excluding the feature flag yaml would keep it from being considered enabled -- though, I think the chatop commands can create one at any time still, which would probably cause an exception further down the chain.
I'm thinking on what gitlab should implement here.
Gitlab should also switch to using the should_track?
method as well.