[track_unique_test_cases_parsed] Enable unique test case parsing tracking
What
Remove the :track_unique_test_cases_parsed
feature flag. This flag enables the usage ping tracking for counting unique test cases with Redis HLL.
Owners
- Team: grouptesting
- Most appropriate slack channel to reach out to:
#g_testing
- Best individual to reach out to: @iamricecake @rickywiens
Expectations
What are we expecting to happen?
We begin to count and track unique test cases across gitlab dot com and self managed instances.
The processing time for unit test reports will increase slightly.
Calls to redis (PFADD
, EXPIRE
) will increase on junit test report parsing. This may increase a lot if the test file being parsed is large. We are doing ~300 keys per Redis Multi command. So a file containing 200k tests, will make 200k/300=667 calls to redis.
On usage ping calculations, there will be an additional call to PFCOUNT
to count the unique test case HLL.
What might happen if this goes wrong?
If the calls to redis are too slow or too frequent the redis instance could experience difficulties.
What can we monitor to detect problems with this?
Beta groups/projects
We will reach out to several smaller projects to enable this feature for them before turning it on for the main gitlab project, and ultimately the gitlab dot com instance.
@zeffmorgan do you have any good ideas as to which smaller projects, using junit reports, could help us test this feature?
Roll Out Steps
-
Enable on staging ( /chatops run feature set feature_name true --staging
) -
Test on staging -
Ensure that documentation has been updated -
Ensure !44473 (merged) is merged -
Enable on GitLab.com for individual groups/projects listed above and verify behaviour ( /chatops run feature set --project=gitlab-org/gitlab feature_name true
) -
Coordinate a time to enable the flag with #production
and#g_delivery
on slack. -
Announce on the issue an estimated time this will be enabled on GitLab.com -
Enable on GitLab.com by running chatops command in #production
(/chatops run feature set feature_name true
) -
Cross post chatops Slack command to #support_gitlab-com
(more guidance when this is necessary in the dev docs) and in your team channel -
Announce on the issue that the flag has been enabled -
Remove feature flag and add changelog entry -
After the flag removal is deployed, clean up the feature flag by running chatops command in #production
channel
/cc @iamricecake