Migrate Gitlab::Counters::BufferedCounter workload to redis-cluster-shared-state
This issue is a follow-up to &1055 (comment 1691838229).
The buffered counter workload was not migrated due to difficulties in achieving consistency across both data stores due to the write-heavy workload. Buffered counter tracks various attributes and flushes the data to the project_statistics
table periodically.
Instead of a MultiStore
-based approach, we will need to update the BufferedCounter
logic to write to the Redis Cluster after a flush is performed.
Proposal
Draft: gitlab-org/gitlab!140630
The proposed approach is to use a Redis key to indicate a switchover using a feature flag. However, on each commit_increment!
, we also run LUA_FLUSH_INCREMENT_SCRIPT
on the BufferedCounter
class to flush any dangling counter keys.
One caveat to note is that while refreshing, the cutover is prevented since refreshing involves setting the attribute in the db to 0.