Skip to content

featureflag: Detect use of feature flags which are not set up by tests

Patrick Steinhardt requested to merge pks-featureflag-sanity-checks into master

When introducing feature flags, then all tests which hit that feature flag should ideally test the code with both the feature flag enabled and the feature flag disabled. In the past, there's been many cases though where this didn't work out: feature flag tests were forgotten, removed by accident, contexts were overwritten unexpectedly, and other problems which are easy to make as a developer. The end result is less test coverage and thus the possibility for breakage.

We can up our game quite easily though: almost all of our tests use testhelper.Context() to construct contexts, which we also use to automatically inject enabled feature flags. We can piggy-back on this infrastructure to inject a sanity check into the context, too: if a certain key is set in the context, then we can make all feature flag checks fail in case the checked feature flag has not been explicitly injected into the context.

Implement this check in our feature flags code. testhelper.Context() now starts to inject this sentinel value to always enable strict feature flag checking for our tests, where our feature sets code knows to mark injected feature flags as injected. Like this, all tests which hit a specific feature flag which wasn't injected via feature sets will start to panic and thus fail the test.

Merge request reports