Skip to content

Add expiration date for Internal Events metrics based on Redis

Problem

Currently, the weekly Redis keys have no expiration date. Meanwhile, RedisHLL keys expire after 6 weeks. The Redis keys are no longer needed after 5+ weeks and are just wasted storage at that point

Desired Outcome

Weekly redis keys are deleted/expired after a certain time.

Proposed Solution

  1. Set expiry data to 6 weeks from now for all existing weekly Redis keys
  2. Set expiry date to 6 weeks from the moment of being triggered to all new weekly Redis keys

The RedisHLL sets it using this const: https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/usage_data_counters/hll_redis_counter.rb#L9

The Redis weekly keys implementation would need to have the expiry added here: https://gitlab.com/gitlab-org/gitlab/blob/master/lib/gitlab/internal_events.rb#L59

Edited by Sebastian Rehm