Log Feature.enabled? requests

Merged Eduardo Bonet requested to merge 344109-improve-observability-over-feature-flags into master

What does this MR do and why?

Logs every call to Feature.enabled?

Logging when a feature flag is requested and the result of the flag is essential to increase observability over changes in the codebase. With this change, we are now able to:

  1. Identify all feature flags that have been acessed on a request.
  2. Use the correlation_id to cross reference with errors, and identify which errors appear when a flag is enabled or disabled.
  3. Quickly identify errors in the codebase, e.g. wrong actor being passed.
  4. Measure traffic over a feature flag.
  5. Identify feature flags that are repeatedly calling the database (within a for loop for example)

How to set up and validate locally

  1. Enable the feature flag through the console (rails -c): Feature.enable(:feature_flag_state_logs)

  2. Open the feature log, grepping for the key feature_flag_states:

    tail -f log/development_json.log | grep feature_flag_states
  3. Run the GDK app

  4. Load any page

  5. Once the request is finished, the state of the feature flags should be added to the logs:

image

Concerns

This will increase the load on our Logging infra. Some follow ups are planned (allowing only feature flags targeted for the current and previous release), and we don't intend to enable the FF 100%.

MR acceptance checklist

This checklist encourages us to confirm any changes have been analyzed to reduce risks in quality, performance, reliability, security, and maintainability.

Related to #345891 (closed)

Edited by Eduardo Bonet