Skip to content

WIP: Change limited_ids_cached to in memory

Dylan Griffith requested to merge 214440-improve-es-enabled-cache into master

What does this MR do?

This switches from storing the list of ids in a Redis key to storing them in an in memory Set that is stored on a class instance variable. This was done due to the fact that the list of ids was growing very large and the cost of sending it back and forth with Redis and unmarshalling and then looking up the existence of an id in the list was becoming considerable. Now in order to invalidate the cache across nodes we use a last modified timestamp in Redis and thus our traffic back and forth to Redis is much less.

We've also introduced a feature flag here to allow us to quickly disable the cache if it causes problems. Incorrect caching has in the past been a big problem and having the ability to quickly disable in production can really help solve problems or debug whether or not a problem is caused by the cache.

See #214440 (closed) for more information.

Screenshots

Does this MR meet the acceptance criteria?

Conformity

Availability and Testing

Security

If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:

  • Label as security and @ mention @gitlab-com/gl-security/appsec
  • The MR includes necessary changes to maintain consistency between UI, API, email, or other methods
  • Security reports checked/validated by a reviewer from the AppSec team

#214440 (closed)

Edited by 🤖 GitLab Bot 🤖

Merge request reports