Skip to content

Use Redis keys with hash-tags Gitlab::ManifestImport::Metadata

Problem

The current implementation of Gitlab::ManifestImport::Metadata uses 2 Redis Strings manifest_import:metadata:user:#{user.id}:group_id and manifest_import:metadata:user:#{user.id}:repositories.

When saving details, it would require 2 setex commands to different keys within a transaction. This is not compatible with Redis Cluster since the 2 keys are likely to be in different key slots.

Proposal

A possible solution to circumvent this cross-slot issue would be to use a Redis Hash so that only 1 key per user id exists.

# current method
MULTI
set manifest_import:metadata:user:#{user.id}:repositories xxxx EX zzz
set manifest_import:metadata:user:#{user.id}:group_id yyy EX zzz
EXEC

# proposed method
MULTI
hset manifest_import:metadata:user:#{user.id} repositories xxxx group_id yyy
expire manifest_import:metadata:user:#{user.id} zzz
EXEC

# alternative method -- SELECTED
MULTI
SET manifest_import:metadata:user:{#{user.id}}:repositories xxxx EX zzz
SET manifest_import:metadata:user:{#{user.id}}:group_id yyy EX zzz
EXEC

We opted for the alternative method as it would introduce the least amount of performance change. The primary focus is to be redis cluster compatible. There are a number of unknowns with regard to Redis Hash performance trade-offs as one of the fields in the hash would be fairly large. The performance gains of using a hash are only applicable if the hash has a small number of keys and the fields are small. See !124871 (comment 1452385455)

Implementation details

In 16.2

  • MR1: Add the dual write to both Redis old and new keys (controlled by feature flag)
  • MR1: Read from new keys with fallback to old keys (controlled by feature flag)
  • MR2: Release the feature by removing the feature flag

MR3 (slightly unrelated): For Redis-Cluster-compatiblity in gitlab-com/gl-infra/scalability#2193, we will need to add a cluster? to skip the cross-slot writes after MR2.

In 16.3 or future release

  • MR4: Remove the dual-write logic in save using manifest_import:metadata:user:#{user.id}:{field} keys.
  • Note: There is no Redis data clean-up required as all keys will expire naturally.

In a +1 release from MR3

  • MR5: Remove fallback reads

See gitlab-com/gl-infra/scalability#2193

Edited by Sylvester Chin