Skip to content

Use GitLab feature flag feature

Investigate using https://gitlab.com/help/user/project/operations/feature_flags that was just released.

Initial configuration

  • gitlab.yml/gitlab.rb? ApplicationSetting? or Environment variables?
  • Unleash.app_name should be distinguished by dev.gitlab.org, staging.gitlab.com, canary.gitlab.com, gitlab.com
  • Which GitLab project should be set as config.url? A project in ops.gitlab.net, gitlab.com, others?

Flexibility - Gate vs Strategy

Optimization

Fetching

  • ETag cache seems not working yet. With it, unleash-client can skip fetching thus we can reduce network I/O.
  • Unleash-client fetches flag values per 15sec by default (polling). => It's supported by default.

Reading

  • Flipper suports L1/L2 cache. Does Unleash-ruby-client need to support it?
  • A recent incident that FF contributed to performance degradation => production#928 (comment 187441674)
  • Each Unleash.is_enabled? walks thgough strategies and the computed values are not cached yet. i.e. If the same flag is read multiple times in a single thread, the computation happens everytime.
  • Should unleash-client cache computed flag into Gitlab::SafeRequestStore (per-request global ivar)?
  • Flipper automatically memoize requested flag statuses (flip.memoize = true, maybe per-request memoization)

Control/Chatops vs UI

Monitoring/Logging

Resiliency/Fallback plan

  • If unleash-client pressurizes the production load and SRE judged we should turn if off immediately, how can we turn it off and fallback to the existing behavior? Feature.enabled?(:unleash_server_enabled) seems necessary.
  • When the polling thread of unleash-client died, how can we recover it without restarting the entire Rails fleet?

Evaluation plan

Documentation/Education

HA

  • As long as it's stick with GitLab-Rails, it's automatically HA.

Geo

  • If master and slaves looking to the same GitLab as a Unleash server, updated values are automatically synchronized in all nodes (because of polling).

On-prem/Omnibus GitLab

Transition period

  • How do we handle existing Flipper-FF? Should we migrate?`

How the system checks a feature on/off with unleash

sequenceDiagram
    participant postgres
    participant unleash server
    participant unleash client
    participant global var
    participant local storage
    participant FeatureA
    loop Polling every 15 sec
      unleash client->>unleash server: Request flags api/v4/feature_flags/unleash/:id
      activate unleash server
      unleash server->>postgres: Retrieving flag data from DB
      activate postgres
      postgres-->>unleash server: Return flag data
      deactivate postgres
      unleash server-->>unleash client: Return flag data
      deactivate unleash server
      unleash client->>local storage: Write flag data as backup file
      unleash client->>global var: Write flag data in memory
    end
    Note left of FeatureA: FeatureA checkes if the flag is on
    FeatureA->>unleash client: Unelash.is_enabled?(:feature_a)
    activate unleash client
    unleash client->>global var: Read flag data from memory
    activate global var
    unleash client->>global var: Return flag data
    deactivate global var
    unleash client->>unleash client: Evaluate with strategies
    unleash client-->>FeatureA: Return ture/false
    deactivate unleash client

How the system checks a feature on/off with flipper

sequenceDiagram
  participant flipper 
  participant ThreadCache
  participant redis
  participant postgres
  participant FeatureA 
  Note left of FeatureA: FeatureA checks if the flag is on
  FeatureA->>flipper:Feature.enabled?(:feature_a)
  activate flipper
  flipper->>ThreadCache:Try to read flag data
  activate ThreadCache
  ThreadCache-->>flipper:Return flag data if exist
  deactivate ThreadCache
  opt if flag data is not found
  flipper->>redis:Try to read flag data
  activate redis
  redis-->>flipper:Return flag data if exist
  deactivate redis
  flipper->>ThreadCache: Cache flag data
  end
  opt if flag data is not found
  flipper->>postgres:Try to read flag data
  activate postgres
  postgres-->>flipper:Return flag data if exist
  deactivate postgres
  flipper->>ThreadCache: Cache flag data
  flipper->>redis: Cache flag data
  end
  flipper->>flipper: Evaluate with gate
  flipper-->>FeatureA: Return true/false
  deactivate flipper

Slack

f_feature_flag

/cc @rspeicher @yorickpeterse

Edited by Orit Golowinski