WIP: Change limited_ids_cached to in memory
What does this MR do?
This switches from storing the list of ids in a Redis key to storing them in an in memory Set that is stored on a class instance variable. This was done due to the fact that the list of ids was growing very large and the cost of sending it back and forth with Redis and unmarshalling and then looking up the existence of an id in the list was becoming considerable. Now in order to invalidate the cache across nodes we use a last modified timestamp in Redis and thus our traffic back and forth to Redis is much less.
We've also introduced a feature flag here to allow us to quickly disable the cache if it causes problems. Incorrect caching has in the past been a big problem and having the ability to quickly disable in production can really help solve problems or debug whether or not a problem is caused by the cache.
See #214440 (closed) for more information.
Screenshots
Does this MR meet the acceptance criteria?
Conformity
- [-] Changelog entry
- [-] Documentation (if required)
-
Code review guidelines -
Merge request performance guidelines -
Style guides -
Database guides -
Separation of EE specific content
Availability and Testing
-
Review and add/update tests for this feature/bug. Consider all test levels. See the Test Planning Process. -
Tested in all supported browsers -
Informed Infrastructure department of a default or new setting change, if applicable per definition of done
Security
If this MR contains changes to processing or storing of credentials or tokens, authorization and authentication methods and other items described in the security review guidelines:
-
Label as security and @ mention @gitlab-com/gl-security/appsec
-
The MR includes necessary changes to maintain consistency between UI, API, email, or other methods -
Security reports checked/validated by a reviewer from the AppSec team