Experiment framework should have key prefix in Redis
Background
During a recent analysis of memory usage in Redis Shared State, we discovered that gitlab-experiment was a large consumer of memory.
Problem
However, it was difficult to identify which keys exactly belong to the experiment framework, because they are not in any way namespaced or prefixed.
We end up with rather generic key names like logged_out_marketing_header
or tier_badge
. In addition to the attribution challenge, this also creates a risk for collisions.
Proposal
As such it would be advisable to migrate these keys to have a gitlab:experiment:
or glex:
prefix, as it makes this kind of analysis much easier.
Migration may need to support both namespaced and non-namespaced keys during the migration phase. groupscalability is able to advise and assist with the process if needed.
To implement this prefix, we'll need to consider this as multiple steps:
-
Add the ability in gitlab-experiment gem to have a prefix configured in config/initializers/gitlab_experiment.rb
.- this will likely mean modifying
lib/gitlab/experiment/cache/redis_hash_store.rb
andlib/gitlab/experiment/cache.rb
and then implementing in GitLab initially with anil
value and then figure out a migration plan.
- this will likely mean modifying
-
Migrate(or not) existing keys to new pattern. - maybe there aren't many experiments at the time of this finishing or all are at 100% or 0% values, so migration will not be required. This will need verified at time of implementation.
-
Change the configured value for GitLab in config/initializers/gitlab_experiment.rb
to have theglex
prefix.- This would likely need either done in a db migration on deployment(it would need to be sure it is executed at right time) or have some sort of deprecation period where things are automatically handled for a bit of time by saying 'if existing use this format, if not use new format' as a monkey patch in GitLab.