feat(datastore): add ability to cache repository objects in Redis
Related to #461 (closed).
This MR modifies the datastore
package to include a new centralRepositoryCache
type that will be our interface to the Redis-backed cache. For now, this will live alongside the existing singleRepositoryCache
, which is being actively used, and this MR deprecates. To reduce risk, we're going to phase out the latter gradually, replacing it with the former.
We'll start by using this new Redis-backed cache for the "get repository details" GitLab v1 API operation only (that will be the next MR after this one). We'll then gradually expand its reach and finally, remove singleRepositoryCache
and rename centralRepositoryCache
to repositoryCache
.
To avoid reinventing the wheel, I've relied on the following libs:
-
https://github.com/go-redis/redis: The Redis client lib. Already a dependency, no surprises here.
-
https://github.com/eko/gocache: A feature-rich Go cache library. There are a lot of great things here. Have a look at the description. The main selling points for me were:
- Compatible with
github.com/go-redis/redis
; - Good code quality/coverage/activity;
- Ability to chain caches. I'd like to have an L1 and L2 cache, where Redis is L2, and an in-memory cache will be L1 (not yet implemented). This will handle and abstract all of that from us;
- Built-in Prometheus metrics collector;
- Includes a marshaller so that we can transparently encode and decode struct to/from Redis. This is done using https://msgpack.org/ (through github.com/vmihailenco/msgpack).
- Compatible with
-
https://github.com/alicebob/miniredis: Pure Go Redis server for tests. This is way easier and more lightweight than spinning up and configuring a Redis server for integration tests.
-
https://github.com/go-redis/redismock: Mocks for
github.com/go-redis/redis
. Together with the above, we should have a solid foundation for testing all Redis-backed features from now on. Have a look at the tests and test helpers introduced here to see what that looks like.